Intro

Good afternoon folks.

Today, I would like to talk a little bit about Cronohub. It’s a Python application which you can use to archive anything from anywhere to anywhere. It uses plugins to archive this versatility.

Let me show you some of its features.

Main Usage

Cronohub is a python application which uses the power and ease of usage of Python to give the user a framework. This Framework can then be used to implement concrete functionality in the form of plugins.

The plugins provide the essential working logic for Cronohub. Cronohub itself, is basically a Hub for these plugins.

What can Cronohub be actually used for? Say, you have a gazillion Github repositories you would like to archive to a S3 bucket. Or SCP to another server. Or to ownCloud… You would use cronohub as such:

❯ cronohub -s {github,gitlab} -t {s3,owncloud,scp}

It is this simple. What’s going on here then? -s tells Cronohub to use a source plugin called github where as -t tells it to use a target plugin called s3 or owncloud or an scp operation.

Under the hood

What happens then is that the source plugin downloads the repositories for a given user. The plugins themselves can require certain configuration options to be provided. Like environment properties. To get the Help of a plugin you can simply ask Cronohub like this:

❯ cronohub help --source-help github
Help (github source plugin):
    - Environment Property:
        CRONO_GITHUB_TOKEN: a token with access to listing repositories for a given user.
    - File that filters the list of repositories to archive. If not present, all will be archived.
        ~/.config/cronohub/configurations/github/.repo_list

It will display help information for that plugin.

Once the source plugin finished downloading the requested repositories to a given location it will pass on a list of files to the target plugin for archiving. The target plugin takes this list of files and will add a timestamp to the file and upload them with the requested operation.

The plugins take care of parallelization or threading if this makes it faster. For example the github plugin downloads at most 5 archive files concurrently.

Plugins

What are plugins then? The plugins adhere to an ABC. An Abstract Base Class. This class defines the abilities and structure of a plugin. It looks like this:

from abc import ABCMeta, abstractmethod


class CronohubSourcePlugin(metaclass=ABCMeta):
    """
    This is the basic definition of a CronoHub plugin.
    """
    @abstractmethod
    def validate(self):
        ...

    @abstractmethod
    def help(self):
        ...

    @abstractmethod
    def fetch(self):
        """
        Returns a tuple (str, str) where there first parameter is the name of the
        archive and the second is the location as a full path. Exp:
        ("my-project-12345", "/home/user/projects/my-project/my-project.tar.gz")
        """
        ...

Validation will be called before the plugin can be used. This method can be used to validate settings for a plugin, for example if a token is provided for the github plugin. Or a bucket name is defined for the S3 plugin etc.

Help will display information just like we saw above and fetch will actually perform the downloading or fetching of files to later archive.

Plugins are located in this repository: Cronohub plugins.

Hopefully, at some point I’ll finish https://cronohub.org and then there will be an online repository for these.

Plugins are located at ~/.config/cronohub/plugins/{target,source}. Each plugin must take care of its own dependencies via a requirements.txt file.

Right now, there are only a few plugins available.

More are hopefully on the way. Local is simply gathering a list of files from a folder. And no-op is what it says. It’s good for debugging a source plugin as it doesn’t do anything else but displays the files it got from the source plugin.

Installation

Installing is dead trivial. Simply run:

pip install cronohub

… and that’s it. It’s published on pypi.org.

Contribution

Plugin contributions are heartily welcomed!

And that’s it.

Thank you for reading,

Gergely.