Automatic Metadata Git Push

Shipdriver-based plugins supports pushing metadata to a remote git repository. The goal is to simplify the process for creating pull requests against OpenCPN/plugins.

When enabled, all metadata files created when the plugin is built are replicated to a fork of OpenCPN/plugins set up by the user. This fork can be used to make a PR against OpenCPN/plugins. In particular, there is no need to download metadata files from Cloudsmith in order to make a PR.

1. Configuration

The configuration is documented in the plugins fork configuration page

2. During the build

When the builders are run, the git-push part will

Make a clone of the OpenCPN/plugins fork repository
Add the new metadata file.
Commit and push the change to the auto branch in the OpenCPN/plugins fork

As mentioned, the changes are pushed to the 'auto' branch by default. The branch name can be tweaked using the GIT_BRANCH environment variable in the builder.

If GIT_KEY_PASSWORD or GIT_REPO is missing git-push will not do anything besides some logging.

3. Making the Pull Request (PR) to OpenCPN/plugins

The builders have made commits in the OpenCPN/plugins fork’s auto branch, one for each build. These commits are the metadata files which work with the plugin installer of the main OpenCPN program. In beta and production mode they become part of the beta and master catalogs of plugins in OpenCPN.

3.1. Merging the auto branch

The files in the auto branch needs to be merged into either master, Beta or Alpha branch. When done, the branch could be used to make a PR against the plugins project. The workflow is somehat different if working with master or Beta branch.

Using Alpha is basically the same as Beta, substituting Beta with Alpha.

3.1.1. Using master

First: sync the local master branch with upstream using this guide or on the command line:

$ git remote update upstream
$ git checkout master
$ git rebase upstream/master

Add the files in the auto branch to master using:

$ git fetch origin auto:auto
$ git checkout auto
$ git rebase master
$ git checkout master
$ git merge --squash auto
$ git commit -m "Merging auto branch into master"

The rebase operation should not be strictly necessary, but will pick up possible errors if the metadata has been changed by other means. The --squash option lumps the too many commits in the auto branch (one for each build) to a single one, keeping the master branch tidy.

A PR (pull request) can then be made from the master branch for updating the master catalog of OpenCPN/plugins.

3.1.2. Using Beta

First: sync the local Beta branch with upstream using this guide or on the command line:

$ git remote update upstream
$ git checkout --track origin/Beta
$ git rebase upstream/Beta

Add the files in the auto branch to Beta using:

$ git fetch origin auto:auto
$ git checkout auto
$ git rebase Beta
$ git checkout Beta
$ git merge --squash auto
$ git commit -m "Merging auto branch into Beta"

3.2. Cleaning up old metadata

Some metadata for older versions is "frozen". This means that these versions are kept as-is without changes when new versions are built.

However, for other versions normally only the last version should be used. The command

    $ ls -lt metadata/my-plugin*

lists all metadata files for given plugin in date order. Use git rm to remove all older files besides those which are frozen. When completed, commit using for example:

    $ git commit -m "Removing obsoleted metadata."

3.3. Check metadata and push

Create a catalog and check it against the xsd specification:

    $ tools/ocpn-metadata  generate --userdir metadata/ \
        --destfile ocpn-plugins.xml --force
    $ xmllint --noout ocpn-plugins.xml

This xmllint check is not strictly necessary, it’s done automatically on every PR when pushed. However, it shortcuts handling of problems.

When testing is complete or just not deemed necessary: push using
git push origin master or git push origin Beta.

4. Troubleshooting

The ci/git-push script can be run manually for debugging purposes as described in the Local CI build page

If metadata from builds does not show up in the auto branch:

First check the build logs: does the git-push step complete? Last part of a successful git-push message should contain something like
```
To github.com:leamas/plugins.git
   6beead5..8c0a9a9  auto -> auto
```
which means that auto branch is updated to 8c0a9a9 . Remember that commit
In the OpenCPN/plugins clone check that commit:
```
$ git fetch origin
$ git show 8c0a9a9
```
This will display the actual changes to the metadata file. Are they sound?
Checkout the auto branch (git checkout auto) and check the metadata file using some editor. It lives in the metadata directory. If it’s fine: done.
If it’s not as expected: check the log for the metadata file. Assuming this is about o-charts-plugin-raspbian-armhf-10.xml:
```
$ git log --oneline  metadata/o-charts-plugin-raspbian-armhf-10.xml
```
This should give a hint about what has happened to the file after 8c0a9a9

5. Security

The private ssh key created by new-credentials is encrypted using a standard DES alghorithm. There is probably some room to crack this given the fact that part of ciphertext is known. The encryption would be stronger if the header and trailer of the key wasn’t encrypted.

That said, given the context this should be reasonably safe. At least, a separate ssh key is used for this purpose, a key which could be easily revoked.

← Shipdriver template overview