Enable versioning on Hetzner S3 Object Storage buckets

Hetzner’s object storage is just out of beta and it’s stable enough to use for common use cases like backup, static file stoarge and hosting, or using it as OpenTofu S3 Storage Backend.

Unfortunately, some features are not yet available in the UI but will eventually be implemented.

One of these features is bucket versioning, which can only be activated using an S3 client like mc, the minio command line utility.
This post covers the setup of mc and enabling bucket versioning.

Installation

The installation of mc is relatively easy. On my linux machine, I just grab it with curl and store it on my ~/.local/bin/ directory and make it executable. My .local/bin/ directory already exists and is home to some other programs I use or wrote myself. Choose either a directory within your path, or create a new directory with the --create-dirs flag and add it to your path like this:

curl https://dl.min.io/client/mc/release/linux-amd64/mc \
  --create-dirs \
  -o $HOME/.local/bin/mc

chmod +x $HOME/.local/bin/mc
export PATH=$PATH:$HOME/.local/bin

(Make sure to export the $PATH variable in your .bashrc/.zshrc, otherwise you will have to export it every time you want to use mc)

Lastly, make sure you can execute mc by running mc --version. This should show something like

mc version RELEASE.2024-10-08T09-37-26Z (commit-id=cf128de2cf42e763e7bd30c6df8b749fa94e0c10)
Runtime: go1.22.8 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

If you don’t yet have a bucket, follow my post about how to to use Hetzner S3 Object Storage as OpenTofu backend, as it covers the steps to create a bucket and credentials. :)

If you have your bucket set up and credentials generated, you can add an `alias`` for your bucket. Make sure you don’t accidentally have your credentials end up in your shell history. Refer to the documentation of your shell to find out how to temporarily disable it.

Add a provider/project alias to mc

mc saves S3 credentials for you, so you don’t have to remember them. This is done with what it calls an “alias”. Let’s add an alias for the bucket of my last post:

Usually, mc just requires you to add the hostname, access key and secret key, but for hetzner, we also need to tell it the s3 api version and turn path off, so that it uses the bucket name as a subdomain, instead of a directory.

mc alias set hetzner-test-project https://fsn1.your-objectstorage.com GYJTKTEGSZQH0NMYOPIX iKLz5codZqtq5Pyqtjf6wOtp5izBeg5tAJwzlgFY --api "s3v4" --path off

Now that you have your alias setup, you can check it out with mc alias list, and it will show you your configuration along with some defaults, that are already set up:

mc alias list
gcs                 
  URL       : https://storage.googleapis.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v2
  Path      : dns
  Src       : /home/oliver/.mc/config.json

hetzner-test-project
  URL       : https://fsn1.your-objectstorage.com
  AccessKey : GYJTKTEGSZQH0NMYOPIX
  SecretKey : iKLz5codZqtq5Pyqtjf6wOtp5izBeg5tAJwzlgFY
  API       : s3v4
  Path      : off
  Src       : /home/oliver/.mc/config.json

local               
  URL       : http://localhost:9000
  AccessKey : 
  SecretKey : 
  API       : 
  Path      : auto
  Src       : /home/oliver/.mc/config.json

play                
  URL       : https://play.min.io
  AccessKey : Q3AM3UQ867SPQQA43P2F
  SecretKey : zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
  API       : S3v4
  Path      : auto
  Src       : /home/oliver/.mc/config.json

s3                  
  URL       : https://s3.amazonaws.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v4
  Path      : dns
  Src       : /home/oliver/.mc/config.json

As you can see, the access key, secret key and other information are stored in json file in your home directory.

You can then list the buckets for these credentials with mc ls [alias]:

mc ls hetzner-test-project

[2024-10-28 20:49:19 CET]     0B my-hetzner-bucket-x4lk7x/

Adding the bucket name to this command, will show you its content:

mc ls hetzner-test-project/my-hetzner-bucket-x4lk7x/

[2024-10-30 07:08:17 CET] 4.1KiB STANDARD my-infra.tfstate

Sweet. This works, so we can now activate bucket versioning.

Enable bucket versioning

Let’s first check, whether versioning is already enabled:

mc version info hetzner-test-project/my-hetzner-bucket-x4lk7x

hetzner-test-project/my-hetzner-bucket-x4lk7x is un-versioned

It’s not, so we can go ahead and enable it:

mc version enable hetzner-test-project/my-hetzner-bucket-x4lk7x

hetzner-test-project/my-hetzner-bucket-x4lk7x versioning is enabled

Listing versions with mc

Now let’s check, whether versioning works:

mc ls --versions hetzner-test-project/my-hetzner-bucket-x4lk7x

[2024-10-30 07:08:17 CET] 4.1KiB STANDARD null v1 PUT my-infra.tfstate

You can see one version of my-infra.tfstate,together with its creation date and size.

After an update to the OpenTofu state, this looks a little different:

mc ls --versions hetzner-test-project/my-hetzner-bucket-x4lk7x

[2024-11-01 11:28:35 CET] 4.8KiB STANDARD .sDyXwoNffIWn-W5XOToETVlMfcaHQF v2 PUT my-infra.tfstate
[2024-10-30 07:08:17 CET] 4.1KiB STANDARD null v1 PUT my-infra.tfstate

Now you can see both versions, their dates, sizes and even their version ID.

That’s it! Versioning is now enabled for this bucket.

Conclusion

Enabling versioning on a Hetzner S3 bucket is currently only possible with an S3 client like mc. To install it, refer to the minio client installation documentation for more information on this.

There’s a lot more you can do with mc, obviously.

One word of caution: All versions will count towards your data usage. Keeping versions of many files that change regularly might add up and increase cost for your buckets. Keep that in mind and thanks for reading! 😄