Partially sync GitLab repo to GitHub

05 Mar, 2021GitLab, CI/CD

Mirroring a whole git repository from GitLab to GitHub can be simply done by the git clone and git push with --mirror option (steps), or use https://github.com/new/import.

But sometimes we don't want such full-scale mirroring, but only to sync a sub directory from the a private repo on a self-hosted GitLab instance to GitHub.com, to publish the example/documentation pages on the GitHub Pages for example.

Git submodule seems to be one of the solution, but it requires developers to push to both repos which is prone to human errors.

So here's a solution that requires a bit upfront setup, but once it's done, developers can simply push changes to the internal GitLab repo only, and a CI job will do the rest.

Step 1

Generate a new SSH key pair, to be used to push code from GitLab CI to GitHub, read more: https://docs.gitlab.com/ee/ci/ssh_keys/

Here we run the ssh-keygen locally, to generate the key pair in the current directory (not to the default ~/.ssh directory), as it will only be used by the CI job, not your own machine.

And later once it's been uploaded to both GitHub and GitLab, you can safely delete the local key pair files (./id_ed25519 and ./id_ed25519.pub).

ssh-keygen -t ed25519 -f ./id_ed25519 -C "Push code from GitLab CI to GitHub"

Step 2

Copy the content of the public key id_ed25519.pub, and paste it at https://github.com/example-user/example-project/settings/keys/new, and don't forget to check the "Allow write access" checkbox.

01 github deploy keys

Step 3

Go to GitLab project Settings > CI/CD Settings > Variables (/-/settings/ci_cd#js-cicd-variables-settings), create a new variable GITHUB_SSH_PRIVATE_KEY and paste the content of the private key id_ed25519.

02 gitlab variables private key

Create two more variables GITHUB_COMMIT_USER_EMAIL and GITHUB_COMMIT_USER_NAME to be used as the commit author info when pushing code to GitHub.

03 gitlab variables

Step 4

Create a new job in the .gitlab-ci.yml

publish-docs:
  image: buildpack-deps:stretch-scm
  script:
    # Optional if your job image is buildpack-deps:stretch-scm, as it comes with openssh-client
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'

    - eval $(ssh-agent -s)
    - ssh-add <(echo "$GITHUB_SSH_PRIVATE_KEY")
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - ssh-keyscan github.com >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts

    - git config --global user.email $GITHUB_COMMIT_USER_EMAIL
    - git config --global user.name $GITHUB_COMMIT_USER_NAME

    - git clone [email protected]:example-user/example-project.git
    - cp -Ra docs/** example-project/
    - cd example-project
    - git add .
    - git commit -m "example-user/[email protected]${CI_COMMIT_TAG:1}"
    - git push
  only:
    - master

And in this example, any changes in docs directory on the master branch will be committed as $GITHUB_COMMIT_USER_NAME and be pushed to GitHub.


Powered by Gatsby. Theme inspired by end2end.

© 2014-2022. Made withby mdluo.