Back when I bought my Synology NAS in 2012, I picked up 4 Seagate 3TB drives. Yes, they’re the same model (ST3000DM001) that BackBlaze reported about in 2015 since these drives have an unusually high failure rate. I even managed to have one drive fail within the first few weeks, but I didn’t think much of it - just submitted an RMA. Since then, I haven’t worried too much about it and those drives have been running for over six years straight with no issues. Well, until recently anyways…

Hmm. Two drives in my NAS are starting to complain...

They’re also 6 1/2 years old…

… But replacing them costs money.

— Matt (@0x2142) April 27, 2018

Almost a month ago, I got a couple of email alerts from my Synology complaining about bad sectors. I hopped onto the device to check it out, and decided to run some of the extended SMART tests against the array. I already have these tests scheduled to run regularly, but I figured it can’t hurt to run them just in case. As luck would have it, Disk 1 of my array failed it’s SMART test within minutes.

Dear user,
Disk 1 on DS918+ has degraded severely and is in failing status. Please make sure that your data has been backed up and then replace this disk.

Additional disk information:
Brand: Seagate
Model: ST3000DM001-9YN166
Capacity: 2.7 TB
Serial number: XXXXXXXX
Firmware: CC9C

S.M.A.R.T. status: Failing
IronWolf Health Management: -
Bad sector count: 128
Disk reconnection count: 0
Disk re-identification count: 0
SSD estimated lifespan: -

The other three disks completed their tests with no reported issues. Disk 3 did worry me a bit though, because it had a rather high bad sector count (~12,000). I figured I would worry about Disk 1 for now, and once it was re-built I would start slowly working my way through the array. The drives were six years old anyways and I didn’t expect them to last forever. At the same time, I really didn’t want to dump the money to buy all new drives just yet. After doing my own research and talking to a few other people I know, I opted to try out the Western Digital Red drives. These drives are made for NAS systems and include some features that standard consumer drives don’t. My array was also around 65% full, so I figured now would be the best time to start expanding the array. I bought one of the 4TB WD40EFRX drives and had it shipped as quickly as possible. Unlike my previous Synology DS411, the 918+ supports the ability to hot-swap drives (the DS411 required disassembling the chassis). Once my new drive showed up, I pulled out disk 1 and put in the new drive. Logged into the Synology DSM interface and told it to start rebuilding the new drive. Simple enough, right?

Well, as luck would have it, I only got a few hours into the rebuild before I got this wonderful email:

Dear user,

Volume 1 has crashed. The system may not boot up.

At this point I was at work and wondering what the state of my NAS was. Did another drive die during rebuild? Have I lost data? Will I be able to recover any of my stuff? I threw in a ticket to Synology support immediately so I could get their input on this. I got back both good and bad news. The bad news? The volume was not going be repairable, and any attempt to repair disks would fail. However, the good news was that my data was fine, but accessible in a read-only state.

I still had my DS411 lying around, so I decided to make the investment in all new drives. I picked up three additional Red drives and loaded all four them into my DS411. Soon enough I had created a new volume/RAID array and I was ready to copy my data over from the DS918+.

The big question was how exactly to replicate the data. I knew Synology had a few options for NAS-to-NAS syncing/backups, but I hadn’t used them before.

The first option was to use Synology’s HyperBackup application. Unfortunately, this required two things I don’t have. First, it would require me to install Hyperbackup, which wouldn’t work since the volume was read-only (a state which also appeared to kill all non-critical packages). Second, it would require me to have at least double the space on my backup device - Enough to store a full-NAS backup, along with enough to restore that entire backup.

The second option for copying the data was using the Shared Folder Sync - which is the method I ended up using. This allowed me to set up a Synology-managed rsync job between the two NAS devices. I did need to make a configuration change to create the rsync tasks, which I couldn’t do on the crashed volume. However, I took a chance and found out that when I reboot the crashed NAS, the volume would be writable until the volume crashed again. This gave me about ten minutes after startup to quickly create the replication job.

Unfortunately my older DS411 could barely handle the CPU requirements needed to copy the data. It did work, but it took a full week to copy ~5.3TB over a gigabit network. Once that copy was finished, I did a quick double-check to make sure I had everything. Then I went ahead and swapped the new drives into my DS918+ following the same steps I used during my original migration.

Overall I think I got extremely lucky here. The only thing I ended up losing was an iSCSI LUN with a few VMs, but I was able to retain all of my critical data. While I do have all of my NAS data backed up to a cloud service, I wasn’t really happy with the idea of having to download 5.3TB over the Internet to restore my data. Since this event, I’ve configured some additional alerting/monitoring of the drives in my DS918+. Next time I see signs of drive trouble I’ll probably just replace it immediately without thinking twice about it. Hopefully this helps out anyone else who runs into this issue! Let me know in the comments if you have any questions

Back in 2011 I picked up a Synology DS411 NAS, which has proved to be one of the most beneficial parts of my home lab. When I purchased it, I filled it with 4x 3TB drives for a total of 12TB of storage (~8TB usable with RAID5). I use the NAS as an iSCSI datastore for my home ESX hosts, which has helped me to run many more test virtual machines than I could have otherwise. I’ve also been using the NAS as a media server, file server, and a general backup location for everything I do.

The only problem with the DS411 is that the device is now over six years old - which means its processing power just doesn’t keep up with what I need it for today. The device is also reaching its end of life state, so I needed to replace it anyways. For reference, the device only came with a single-core 1.6Ghz processor and 512Mb of RAM.

Synology just recently released their new 2018 models, so I opted to pick up the DS918+. I could have upgraded to an equivalent model(DS418) of what I already have, but I was really interested in some of the additional features offered by the plus-series model. The DS918+ supports docker containers and Synology’s own virtualization hypervisor - along with the ability to add extra RAM modules later if I need them.As sad as I am to see my DS411 go, it was definitely time for an upgrade!

Anyways, I just completed the migration from my old DS411 NAS to a new DS918+. The whole process was much easier than I had anticipated, but I figured I would write up a quick summary of what I did:

1. First thing - Update the current NAS to the latest version of Synology DSM

   • Control Panel > Update & Restore > Click Update if one is available

2. Take a backup of the DSM configuration

   • Control Panel > Update & Restore > Configuration Backup > Click Backup Configuration

3. Power off the old NAS - in my case, my DS411

4. Unplug the old NAS and remove the drives

   • For the DS411, this requires disassembling the chassis
   • MAKE SURE YOU KEEP THE DRIVES IN ORDER (I actually printed labels to put on mine)
     • For the DS411, the drive numbers start top to bottom (Disk 1 is top, Disk 4 is bottom)

5. Install the drives into the new NAS - in my case, a DS918+

   • Again, these drives must be replaced in the same order!
     • The DS918+ numbers left to right (Disk 1 is the first slot on the left, Disk 4 is last slot on the right)

6. Once all of the drives have been inserted - Plug in the new NAS and power it on

7. Download the Synology Assistant application

   • This is necessary because the new NAS will not retain the previous IP configuration of the old NAS

8. Once the new NAS is booted up - Open Synology Assistant and click Search

   • It should locate the NAS, and the status should be Migratable
   • In my case, it shows both network adapters for the NAS

9. Select the NAS and click Connect - This will launch the Web UI of the NAS

10. The WebUI should show something similar to the screenshot below, with the button to Migrate

    • If the WebUI does not show you a migrate option, DO NOT CONTINUE. You may need to double-check that the drives have been inserted in the correct order.

11. Click Migrate, then you will be prompted to select whether you would like to migrate all settings or perform a clean install of DSM

    • Performing a clean install will still retain all of the data on the NAS, but all DSM settings will be lost
    • I selected the option to retain all of my settings

12. Next, Click through the prompts to install the newest version of DSM

13. Wait for the system to download and install DSM

Once complete - you will be brought to your DSM login screen and the migration is complete!

If you selected to keep the DSM settings, everything should still be there - with the exception of your IP/network configuration.

After all that is complete - you’re ready to enjoy your new Synology NAS! The migration was significantly easier than I had expected it to be. The longest part for me was just removing the drives from the DS411 - since it requires disassembling the chassis and removing multiple screws to free the drives from the drive sleds. So far the DS918+ is fantastic - and I would highly recommend purchasing one to anyone who is interested.

Hope this quick tutorial helps out - Let me know in the comments!

As part of my home lab, I have an older Synology DS411 that I picked up in early 2012. I’ve been using the device since then with 4x 3TB drives as an iSCSI backend to my ESX host. Of course, I’ve also been using the NAS for general file storage (photos, videos, documents, etc). So in late 2014 I decided that I needed to find a good backup solution for it.

I started using CrashPlan, because they offer unlimited cloud storage for only $5.99 a month. This was amazing to me, because I had around 1.2TB of data at the time. I found a great community package here, which allowed me to install the backup client directly on the NAS. This ran great until late 2016 when CrashPlan updated to 4.8.0 and ended support for ARM processors (which is what the DS411 uses). For the past 6-7 months, I’ve been unable to back up my primary storage device at home. Since my DS411 is reaching the end of it’s life anyways, I figured I would just wait until I replaced it with one of the new Synology devices that use an Intel processor.

Well a few days ago, I received an email from CrashPlan threatening to delete my backups since my NAS hadn’t connected in over 6 months. It makes sense why they do that - but I didn’t want to lose my backups before I could replace the device. Re-seeding my backups whenever I picked up a replacement NAS would take forever (I think I have ~2TB backed up currently). So I finally forced myself to sit down and find an alternate solution.

Alright - so as of today I now have a dedicated CentOS VM running on my ESX host, which is connected to the Synology via NFS. This VM is running the CrashPlan client, and my backups have resumed! Here is how I got this all set up:

Synology Configuration

1. Enable NFS on the Synology

   • Open up the Control Panel and go to File Services
   • Scroll down to the NFS section, and check the box for Enable NFS
   • Click Apply

2. Apply NFS permissions to each share

   • Still under Control Panel, visit Shared Folders
   • For each folder you need to back up via CrashPlan:
     • Select the folder and click Edit
     • Click the tab for NFS Permissions, then click Create
     • Enter the IP address for your CentOS VM (or other linux system)
     • Set privilege to Read-Only (you could probably leave this read-write, but CrashPlan only needs read permissions to back up data)
     • Click OK

Linux VM Setup

1. Build a CentOS VM - This can be on an ESX host like I used, or just a standalone PC

   • Don’t forget to use the same IP address that we entered in the Synology for NFS

2. Install packages

   • Make sure you get the latest package updates first: yum -y update
   • Install NFS tools: yum -y installnfs-utils nfs-utils-lib

3. Test NFS

   • If you have the packages installed and NFS set up correctly on the Synology, then you should be able to validate the configuration by using showmount. For example, here are the directories that I was using CrashPlan to backup:

   [root@SynologyBackupVM~]# showmount -e 10.12.32.2
   Export list for 10.12.32.2:
   /volume1/backups   10.12.32.209
   /volume1/documents 10.12.32.209
   /volume1/homes     10.12.32.209
   /volume1/photos    10.12.32.209
   

   • Make your local Linux directory structure, which shouldmatch what the Synology structure is. So in my case, I made new directories on my local Linux VM for /volume1/backups, /volume1/documents, etc
   • Edit /etc/fstab to auto-mount your NFS shares on boot. In my case, I added the following lines:

   # NFS Share location          Local Folder
   10.12.32.2:/volume1/backups   /volume1/backups   nfs defaults 0 0
   10.12.32.2:/volume1/documents /volume1/documents nfs defaults 0 0
   10.12.32.2:/volume1/homes     /volume1/homes     nfs defaults 0 0
   10.12.32.2:/volume1/photo     /volume1/photo     nfs defaults 0 0
   

   • Reboot, and check to make sure each NFS share was actually mounted. If you do a ls /volume1/backups, do yousee all the files on the NAS in that folder?
   • If everything works, then grab the CrashPlan Client installer (4.8.2 was the latest at the time)
     • wget https://download.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.8.2_Linux.tgz
   • Extract the files: tar -xzvf CrashPlan_4.8.2_Linux.tgz
   • Run the installer: cdcrashplan-install/ && ./install.sh
     • I just let CrashPlan use all the defaults, so I didn’t change any options during install
   • Set CrashPlan to start automatically: chkconfig crashplan on
   • Make sure the service is already running: /etc/init.d/crashplan status

At this point we should be able to run CrashPlan’s backup client on our VM, which will pull the data across the network from the NAS. The last step is to set up our local PC for remote administration of the Linux VM. Unfortunately, this process has become more and more painful as CrashPlan keeps updating their clients. They provide their own documentation on how to do this piece, but I’ll summarize what I did here:

Connecting the CrashPlan UI to the Linux VM

1. Download and install the CrashPlan Client on your PC (In this case, I’m using a Windows 10 laptop)

2. On your linux VM, run the following command to findyour authentication token: cat /var/lib/crashplan/.ui_info

3. Back on the Windows side, place that token in the following file: C:\ProgramData\CrashPlan.ui_info

   • You will replace the existing token in the file
   • Also change the port from 4243 to 4200
   • Should look something like this when you’re done: 4200,3da4v903-7q38-4r52-e67e-79aecxf760c4,127.0.0.1

4. Download PuTTY, which will be used to create a SSH tunnel to our linux box

5. Open PuTTY and set the following configurations:

   • On the left side, go to Connection > SSH > Tunnels
   • Enter thesource port: 4200
   • Enter the destination: localhost:4243
   • Click Add
   • On the left side, go back up to Session
   • Enter the IP address of your Linux VM
   • (optional) Under Saved Sessions, put in a name and clickSave

6. Once that’s done, go ahead and clickOpen to connect to your Linux VM

7. Log into the VM and just leave the window open.

   • Note: This SSH session will need to remain open any time you need to connect to your Linux VM and administer CrashPlan

8. Open the CrashPlan client locally on your Windows machine

9. Log into CrashPlan using your account (Should be the same one you were previously using to back up your Synology with)

10. CrashPlan may give you a warning about migrating to a new PC, and ask if you want to adopt the backups - You want to acceptthis prompt, and let CrashPlan know that your new Linux VM is a replacement PC for your Synology.

As long as everything went according to plan, the CrashPlan client should start scanning the NFS shares on your Linux VM and comparing them to what’s already backed up. Once it completes its synchronization, it will initiate the backup processes again.

I was extremely happy that this worked, because I was able to start backing up my data again (which had apparently almost doubled since the last time CrashPlan was connected). In my case, moving to a Linux VM provided me with much better backup performance as well, since the Synology DS411 only has a 1.6Ghz single-core processor and 512MB of RAM.

I hope this helps out anyone else out there who may have been trapped in a similar scenario. If you have any questions, please feel free to leave me a comment below!

If you really want to become great at something, you practice it a ton, right? Well networking and IT work exactly the same. You’re not going to become an expect by just reading a ton of tech books and blogs. While those certainly help, there is nothing better than simply getting your hands dirty. Having a good home lab setup is key to truly understanding how things work.

So how do you get started? Well the way that I built a home lab over the past 10 years is probably much different from you could today, given the amount of virtualization technologies available. Still, I believe that some physical pieces of equipment are necessary. I took classes in high school toward CCNA certification, and we had a lab of several routers and switches there. Once I got into the real world, I wanted to start working on additional certifications and just improve my skills overall. So I picked up an old Cisco 2611 router and a 2950 switch. I played with these for a bit and used them to get my CCNA Security, which at the time covered the basics of securing Cisco IOS routers and switches.

Another year or so down the road and I expanded by picking up a power-over-ethernet switch, and two Cisco 7900 series IP phones. Since I had discovered that the 2611 router could run Cisco’s Call Manager Express, I decided to go for the CCNA Voice certification. Having this equipment to work on gave me experience that was much closer to real world, than if I had just studied the textbooks. I could configure things, break things, then sit there for hours until I figured out how to fix my problem. I could configure the entire system, test it all, then tear it down and completely rebuild. Being able to configure the entire CME system from memory gave me a lot of confidence toward taking the certification exam.

So do I still have a home lab today? Oh yeah, you bet I do! It’s changed quite a bit from what it used to be, but the same concept still applies. I have an entire environment to play with, which allows me to test and learn new technologies outside of work. In fact, my ‘home lab’ has evolved into just part of my home networks. So here is what I’ve got running today:

• Cisco ASA 5505 (Probably soon to be replaced with a Juniper SRX 300)
• Two Cisco 2960G-8TC-L switches
• Ubiquiti UniFi 802.11n wireless access point
• Synology DS411 Network Attached Storage with 4x 3TB drives (Soon to be replaced, as it is over 5 years old(Update: Got myself a DS918+!))
• A few spare PCs running VMware ESX 6.0

The ASA, switches, and AP run just about all of my home network. I even have the ASA running AnyConnect SSL VPN so I can access my storage at home from anywhere. The Synology has been one of the best additions to my network and lab. For one, it acts as a centralized storage device for my home network. I back up all of my PCs to it, and any digital media I own is also stored on it so I can stream it to devices within my home. For two, the Synology acts as an iSCSI backend to my VMware hosts. This setup allows me much more flexibility with my lab.

On the ESX hosts, I have a few VMs for lab use and a few that are for my home network. A GitLab server hosts all of my Git repositories for my own personal coding projects. I have a CentOS box for running the Ubiquiti management web interface. Another few CentOS VMs for running bind DNS, Observium, and Splunk. I also run a personal Minecraft server on there, so it’s not all work here 🙂

I love the idea that at any time I can just go home, spin up a few VMs, and start playing with something new. When I was learning Juniper’s SRX platform, I downloaded their free trial of the vSRX and had it running for a while. When I changed jobs, I needed to learn a new web proxy software - so I downloaded their free trial and stood up a VM. You really learn a lot by building a platform from scratch, because you gain a better understanding of what impact certain configuration options have. You also have the freedom to change whatever settings you want and see what they do. I once had an idea for a coding project, so I turned up a VM running RabbitMQ - and spent a weekend learning how it works to see if it would accomplish what I needed for the project.

So to sum it up - I just want to say that having a home lab has really contributed a lot to my success. It offers way more flexibility than trying to test something at work, unless they also offer you a complete lab environment. Your lab doesn’t have to start off perfect, nor does it need to have expensive equipment - it just needs to help facilitate your ability to learn and gain experience. Have a lab at home? Tell me about it in the comments below! I would love to hear what other people have done.