Getting off Big Tech: Proprietary cloud alternatives

Despite spending many years manually copying photo folders from hard drive to hard drive, I’ve gotten quite used to the convenience of accessing and making changes to my photos, calendars, and contacts on all devices. However, when I started to become frustrated with the amount of control I had slowly given big corporations over my life, (see The Future of Tech: Building sustainably with Open Source), I started looking for alternatives. After over a year of research, trying and using various Open Source services, it feels like a good time to share my current suite of services and walk through their pros and cons.

I chose to setup and maintain a home server to host these services, but many of them offer paid managed instances that you can simply sign up to use. Yes, you would still be storing your data with another company, but the code they are running is Open Source, so you know exactly how your data is stored and used. Be sure to read my previous post for details about why self-hosting or using managed services might be better for you.

Nextcloud

Nextcloud is essentially an open source Google Drive or Dropbox, with support for multiple accounts, file sharing between accounts, and public share links with various levels of permissions (view, write, or upload-only). It also supports calendars, contacts and tasks, and has a nice ecosystem of “applications” that integrate with it and greatly increase its capabilities—such as Kanban boards, notes, photo gallaries, and much more. Personally, while it was initially temping for me to try to use as many of these as possible to share one account, I found that there were often better standalone alternative services.

For instance, as a photo or video storage solution, I’ve found that Nextcloud struggles to efficently render and show previews. It doesn’t play video smoothly through the UI. It also doesn’t have the kind of searching or display features that I’d want in a media gallery, like searching by location, dates, people, and contents of images. I find it’s best suited for non-media type files.

Syncing of calendars, contacts, and tasks works really well with the DAV API that integrates with my Mac and Linux native apps, as well as my phone using the DavX App. Additionally, there is a Nextcloud desktop tray app that works very similary to Dropbox or Google Drive, allowing you to 2-way sync all or some of your Nextcloud drive to a folder on your computer. The Nextcloud Android app allows you to browse your drive and auto-upload newly created files in configured folders to Nextcloud. You can configure it to upload photos to a “Camera Roll” folder on Nextcloud. However, this is only a one-way sync, so deleting a picture on Nextcloud won’t sync back to your phone. Deleting the photo on your phone won’t trigger a “delete” on Nextcloud since it only uploads new files. It isn’t a true “sync.”

There are alternative apps that allow the 2-way sync, like FolderSync. FolderSync is a proprietary app that you can pay for or use with ads. So far, I choose to not use this, since I don’t really want to give my personal cloud credentials to a proprietary app.

If you are running this on your own server, I had issues with object storage like S3 and ran into a bug. While this has since been fixed, my overall impression is that S3 storage is a bit behind in features and stability.

Nextcloud also has many managed instances that you can just sign up for. Most of them have a free tier if you just want to try it out for calendars, contacts, and a few GB of data.

Nextcloud is well maintained and optimized, but it is written in PHP to maximize the kind of hosting options available to admins. But it isn’t the fastest language option out there, and in my opinion, you can feel it a bit. It’s definitely not as snappy to navigate as Google Drive, but the features are rich. So far, it’s served as a fantastic Google Drive replacement.

OnlyOffice

OnlyOffice is a desktop app replacement for something like Microsoft Office. It is also a collaborative cloud based replacement for Google Docs. You can run it standalone on a server, but I have it running only internally-facing on my server, and integrate it with Nextcloud. When I open any office file from the Nextcloud interface, it opens in the OnlyOffice editor embedded in Nextcloud. It has almost complete feature parity with Microsoft Office, allowing you to read and write Microsoft formats or use the open standards.

I don’t do a lot of “Office” work, but whenever I’ve needed it, it’s worked flawlessly, from writing blog posts to presenting live. The OnlyOffice Desktop app also allows you to login to your Nextcloud instance and edit remote files using the native app. It also support collaborative editing just like Google Docs, and implements dark mode!

The desktop and mobile apps are free. It’s also free to host your own cloud instance, but many of the Nextcloud managed services also include OnlyOffice in the cloud.

PhotoPrism

PhotoPrism is basically a replacement for Google Photos, with support for comprehensive search tools using date/time, locations, contents of the photo, faces, etc. You can view and interact with photos on a map. It extracts detailed information from the file’s EXIF data. It also has a machine learning model that analyzes all your photos for common contents like “dog,” “interior,” and “boat,”. It also does facial recognition. So just like Google Photos, if you are trying to find an old photo and don’t remember when or where you were, but remember it was on a farm with cows, search “cows” or “farm” and you might just find it.

The UI is very modern and sleek. It’s written in GoLang so it’s very performant. The previews load and scroll smoothly and video playback is clean and unobtrusive.

There is still one major downside, but only because the project is still very new and not feature complete. At the moment, it doesn’t support multiple users. And sharing is limited to sharing a view-only public link to an album.

This means that none of my family can use it yet. It’s just for me until this feature is complete. This also means that there aren’t any managed versions of PhotoPrism out there that you can just sign up for. You have to run it yourself.

With that, I was left with two choices:

1. Let PhotoPrism manage all of my photos, removing them from Nextcloud, so that I can upload and delete photos directly from the PhotoPrism UI.
2. Let Nextcloud continue to manage my photos, and simply use PhotoPrism as a read-only instance to my user’s Photos folder.

Option 1 means I get a nice all-in-one experience to managing my photos. But it also means I need to find a new photo sync mechanism, using the DAV API provided by Photoprism and probably relying on the proprietary FolderSync app. It also means that I wouldn’t have access to many good photo sharing options. I routinely share albums between me and my dance partner, or family members. Without multi-user support and sharing, this isn’t possible. I also often create public “photo drop” links to allow people to give me photos or videos they may have taken of me at one of my ballroom dance competitions.

Option 2 allows me to have all of my existing sharing and syncing features. But it means that I can’t use the PhotoPrism interface to upload, or more importantly, delete photos directly. This means, if I’m scrolling through my library and want to clean out some duplicate or low quality images, I need to go find that image from the Nextcloud interface and delete it.

For now I’m going with option 2, because there are no deal-breakers like option 1, just a few inconveniences. Once they support multi-user functionality, I will probably migrate all my photos over to be managed directly by PhotoPrism since it’s just a better experience. And I’ll probably spend some more time looking for an Open Source DAV File Sync app for my phone, maybe even look into developing one.

Matrix chat server with Synapse & Element

Matrix is a well-established protocol for federated messaging. Synapse is a python implementation of this protocol, and Element is a front-end client for a matrix service. It is a very mature and polished replacement for services like Facebook Messenger, WhatsApp, Slack, and even Zoom (integrated for 1-1 videos, Jitsi integration for group calls). Element supports true end-to-end encryption, though I’m not sure it’s entirely worth the extra complexity when all of the data never lands on any server I don’t own. It’s a polished enough experience that I’ve been able to get all of my family and close friends on the platform, keeping all of my conversation data out of corporate hands.

Nextcloud does have a “Talk” app that allows chat and video call functionality, but the feature set is pretty minimal. The UI is also not nearly as polished as Element.

If you are content using the Element desktop and mobile apps, you don’t need to host your own web based element interface. But I chose to host that as well so it’s easy for my friends and family to get started. They can simply access Element at chat.mydomain.com.

Because it supports federation, you can actually chat with users who aren’t on your server. For instance, I can join an “Elixir” channel on the matrix.org instance. All of the data in that channel with be “federated” and replicated on my server and on the matrix.org server. Basically, servers will only store conversations where their users are participating, unlike Facebook Messenger which has access to everyone’s conversations.

The experience is not flawless, especially when it comes to end-to-end encryption. Sometimes messages fail to decrypt after an update, requiring you to log out and back in, but BE CAREFUL. If you log out without having backed up your encryption keys, or aren’t logged in on another device, you’ll lose all your conversation history. Reloading your encryption keys from another logged in device requires a few back-and-forths, scanning QR codes and can be a bit confusing to non-technical people.

I would say the major downside to running or using Matrix is that it can be hard to get buy-in from your friend group. And since a messaging app is useless by yourself, that is a pretty big deal. And good luck trying to figure out whether to call it “Matrix”, “Synapse”, or “Element” with your friends. But with a little patience, they might come around!

Synapse also doesn’t really have an admin UI. Since I didn’t want my server to be open to everyone, I configured the instance to not allow user sign up. But I then had to create users manually from scripts in the Docker image. Later I set up the matrix-registration service which allows me to create tokens that I can give to friends, allowing them to sign up themselves if they have the token.

Wireguard VPN

While I don’t have constant need for a VPN, it can sometimes be useful for me when I travel abroad and want to access services that are in the US. For instance, one time I was trying to order a cellular hotspot for my family, but the service that the phone company used to assign a phone number was giving me a 403 Forbidden while I was out of the country. I turned on my VPN and continued without issue. It is also useful for doing client coding work to make sure I keep client code safe on US networks.

Wireguard is one of the newer VPN protocols. It is incredibly fast–you will never notice you are connected–and stays connected seemlessly, even when you switch between cellular data and WiFi.

A lot of companies out there like NordVPN use Wireguard under the hood, but I would probably use something like Mozilla VPN if I were looking for a managed Wireguard service to sign up for. They are a non-profit, and I have a vested interest in helping them keep Firefox alive to maintain an Open Web.

Obsidian

I don’t feel like I’m the right person to describe what Obsidian is so I’ll quote their home page.

Obsidian is a powerful knowledge base on top of a local folder of plain text Markdown files.

Currently I use it as my note taking app, but I know that is a vast underuse of this powerful software. Obsidian is somewhat like the “Vim” of note taking apps, super modular and customizable, encouraging you to find your own optimized workflow. It includes some special “linking” syntax that allows you to link common ideas throughout your notes. It builds a graph of these ideas over time. From this graph you can connect thoughts over time and form new ideas.

There are all kinds of theories and workflows on how best to use Obsidian, such as the Zettelkasten Workflow. I hope to start utilizing this more in the future. I have to admit, I mostly chose this app because it has a beautifully designed UI for both the desktop and mobile.

Obsidian does have a paid syncing capability, but I simply rely on the Nextcloud tray application to sync up my Obsidian Vault like it does any other files on my computer. On the phone side, it’s not so simple since the Nextcloud app doesn’t support actual 2-way sync. The recommended solution is to use the FolderSync app. But as mentioned earlier, I don’t really want to give a proprietary app access to my private cloud credentials. At the moment I just sync the vault manually using an rclone utility app called RCS, but this isn’t my ideal final solution.

Conclusion

As you can see, this process isn’t complete. I’ve definitely made some tradeoffs, some of which I need to put more time or work into improving. Others I just accept. Your mileage may vary.

There’s definitely a lot of overlap between functionality of various services. But in general, I’ve found that it’s best to embrace the Unix philosophy of “small single-purpose over monolith.” I run many separate services that were developed specifically for that purpose, rather than trying to extract the maximum functionality out of one service like Nextcloud. This comes at the cost of lack of integration. It would be nice if I could use one account for Nextcloud, Matrix, and PhotoPrism, but currently I don’t. Some of these services can support SSO or OAuth logins. Perhaps at some point I could spin up an OpenSource SSO service and wire it all together.

In general, being the one person in a friend group that does everything the janky way doesn’t add to your popularity. But I’ll say, the one time that person says, “Wow, thanks for setting this up, it’s so cool that we can have this conversation on a completely non-profit platform,” it makes it all worth it.