Structural changes

Swap order of PAT / SSH keys in troubleshooting. 

Add error sections to troubleshooting. 

Clarify points in text.
This commit is contained in:
jen-thomas 2022-09-05 10:12:45 +01:00 committed by GitHub
parent 36c541c94b
commit 50fe61c592
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

168
README.md
View File

@ -1,10 +1,10 @@
# github-action-push-to-another-repository # github-action-push-to-another-repository
This GitHub Action will help to push files that exist in a GitHub repository, or that are generated by another GitHub action, into a new repository. This GitHub Action will help to push files from one GitHub repository to another. For example, on a git push in a `source` repository, files can be pushed into a `destination` repository.
For example, on a git push in a `source` repository, you want certain files to be pushed into a `destination` repository. A couple of examples of this could be: A couple of examples of this could be:
* Specifying a directory in the `source` to be pushed to `destination` * Specifying a directory in a `source` repository to be pushed to a `destination` repository
* Generating some files using another GitHub Action (e.g. MarkDown to PDF) and pushing the PDFs across * Generating some files using another GitHub Action and pushing them into another `destination` repository (e.g. generate PDFs from MarkDown and push the PDFs across to the `destination` repository)
Please read the documentation to understand which files and commits are kept in the destination repository and `target-directory`. Please read the documentation to understand which files and commits are kept in the destination repository and `target-directory`.
@ -12,23 +12,22 @@ Please read the documentation to understand which files and commits are kept in
## Table of contents ## Table of contents
* [General flow](#general_flow) * [Overview](#overview)
## General flow ## Overview
There are two examples: There are two ways to set up this GitHub action:
* Using SSH deploy keys (recommended, a bit harder to set up): [push-to-another-repository-deploy-keys-example](https://github.com/cpina/push-to-another-repository-deploy-keys-example). The configuration is in the file [.github/workflows/ci.yml](https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/.github/workflows/ci.yml#L21) * Using SSH deploy keys (recommended, a bit harder to set up): [push-to-another-repository-deploy-keys-example](https://github.com/cpina/push-to-another-repository-deploy-keys-example). The configuration is in the file [.github/workflows/ci.yml](https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/.github/workflows/ci.yml#L21)
* Using a Personal Access Token (first iteration, not recommended but easier to set up: [push-to-another-repository-example](https://github.com/cpina/push-to-another-repository-example). The configuration is in the file [.github/workflows/ci.yml](https://github.com/cpina/push-to-another-repository-example/blob/main/.github/workflows/ci.yml#L21) * Using a Personal Access Token (first iteration, not recommended but easier to set up: [push-to-another-repository-example](https://github.com/cpina/push-to-another-repository-example). The configuration is in the file [.github/workflows/ci.yml](https://github.com/cpina/push-to-another-repository-example/blob/main/.github/workflows/ci.yml#L21)
In the above examples, the file [build.sh](https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/build.sh) is executed, which creates a new directory `output/` and this directory is copied across to the [destination repository](https://github.com/cpina/push-to-another-repository-output) Once the setup using one of the methods above has been done, the file [build.sh](https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/build.sh) is executed, creating a new directory `output/`, and this directory is copied across to the [destination repository](https://github.com/cpina/push-to-another-repository-output).
Before using the GitHub Action, **either the Personal Access Token or the SSH Keys need to be set up as described below.**
:warning: Please bear in mind that the files in the specified directory of the target repository are deleted unless the option `target-directory` is used (in this case, only the files for this directory are deleted). :warning: Please bear in mind that the files in the specified directory of the target repository are deleted unless the option `target-directory` is used (in this case, only the files for this directory are deleted).
There are different variables to set up the behaviour:
## Usage ## Usage
The variables below set up the behaviour of the GitHub Action:
### `source-directory` (argument) ### `source-directory` (argument)
Directory that the GitHub Action will push files from. Note: it can be `.` to push all the repository, but read the FAQ! Directory that the GitHub Action will push files from. Note: it can be `.` to push all the repository, but read the FAQ!
@ -61,16 +60,20 @@ The string `ORIGIN_COMMIT` is replaced by `$ORIGIN_REPOSITORY_URL@commit`.
The directory to wipe and replace in the target repository. Defaults to wiping the entire repository. The directory to wipe and replace in the target repository. Defaults to wiping the entire repository.
## Setup ## Setup
Before using the GitHub Action, **either the SSH Keys (recommended) or the Personal Access Token need to be set up as described below.**
### `SSH_DEPLOY_KEY` Vs. `API_TOKEN_GITHUB` ### `SSH_DEPLOY_KEY` Vs. `API_TOKEN_GITHUB`
The action, entirely executed in your GitHub continuous integration environment, needs to be able to push to the destination repository.
The Action, entirely executed in your GitHub continuous integration environment, needs to be able to push to the destination repository.
There are two options to do this: There are two options to do this:
* Create an SSH deploy key. This key is restricted to the destination repository only * Create an SSH deploy key. This key is restricted to the destination repository only (recommended)
* Create a GitHub Personal Authentication Token. The token has access to all your repositories * Create a GitHub Personal Access Token. The Token has access to all your repositories
Someone with write access to your repository or this action, could technically add code to leak the key. Thus, *it is recommended to use the SSH deploy key method to minimise the impact* if this were to happen. Someone with write access to your repository or this Action, could technically add code to leak the Token. Thus, *it is recommended to use the SSH deploy key method to minimise the impact* if this were to happen.
This action supports both methods to keep backwards compatibility, because in the beginning it only supported the GitHub Personal Authentication token. Initially, this Action only used the Personal Access Token method and it still supports both methods to keep backwards compatibility.
### Setup using SSH deploy key ### Setup using SSH deploy key
@ -78,8 +81,8 @@ Recommended, but the setup has a few more steps compared with the Personal Acces
#### Generate the key files #### Generate the key files
* In your computer terminal, generate an ssh key using: `ssh-keygen -t ed25519 -C "your_email@example.com"` (the type ed25519 is recommended by [GitHub documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key)). * In your computer terminal, generate an SSH key using: `ssh-keygen -t ed25519 -C "your_email@example.com"` (the type ed25519 is recommended by [GitHub documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key)).
* ssh will ask for a file path: `Enter file in which to save the key`: write a new file name. I suggest the default directory and as a filename: `id_github_{name_of_your_destination_repository}` to avoid overwriting a previous file. If you will be using this action for multiple repositories, you might want to generate different keys for each one. For the repository `https://github.com/cpina/push-to-another-repository-example/`, `id_github_push-to-another-repository-example` could be used. * `ssh-keygen` will ask for a file path: `Enter file in which to save the key`: write a new file name. I suggest the default directory and as a filename: `id_github_{name_of_your_destination_repository}` to avoid overwriting a previous file. If you will be using this Action for multiple repositories, you might want to generate different keys for each one. For the repository `https://github.com/cpina/push-to-another-repository-example/`, `id_github_push-to-another-repository-example` could be used.
* Leave the passphrase empty (otherwise the GitHub Action cannot use it) * Leave the passphrase empty (otherwise the GitHub Action cannot use it)
Example: Example:
@ -111,7 +114,7 @@ The key's randomart image is:
carles@pinux:~$ carles@pinux:~$
``` ```
This has generated two files, the public and private ssh keys: This generates two files, the public and private ssh keys:
``` ```
carles@pinux:~$ ls -l /home/carles/.ssh/id_github_push-to-another-repository* carles@pinux:~$ ls -l /home/carles/.ssh/id_github_push-to-another-repository*
-rw------- 1 carles carles 411 Jul 28 09:40 /home/carles/.ssh/id_github_push-to-another-repository -rw------- 1 carles carles 411 Jul 28 09:40 /home/carles/.ssh/id_github_push-to-another-repository
@ -123,7 +126,7 @@ carles@pinux:~$
#### Destination repository #### Destination repository
In this section we will add the generated public key to the destination repository. This allows the Action to push there. Text instructions: In this section, we will add the generated public key to the destination repository. This allows the Action to push there.
* Go to the GitHub page of the **destination** repository (e.g. https://github.com/cpina/push-to-another-repository-output) * Go to the GitHub page of the **destination** repository (e.g. https://github.com/cpina/push-to-another-repository-output)
* Click on "Settings" (settings for the repository, not the account settings) * Click on "Settings" (settings for the repository, not the account settings)
@ -144,10 +147,11 @@ In this section we will add the generated public key to the destination reposito
![Screenshot-title-key-enable](docs/screenshots/ssh-key-40.png) ![Screenshot-title-key-enable](docs/screenshots/ssh-key-40.png)
#### Source repository
#### Origin repository In this section, we will add the generated private key to the source repository. This allows the Action to push to the destination repository.
* Go to the GitHub page of the origin repository (e.g. https://github.com/cpina/push-to-another-repository-deploy-keys-example) * Go to the GitHub page of the source repository (e.g. https://github.com/cpina/push-to-another-repository-deploy-keys-example)
* Click on the "Settings" (settings for the repository, not the account settings) * Click on the "Settings" (settings for the repository, not the account settings)
@ -168,13 +172,13 @@ In this section we will add the generated public key to the destination reposito
The GitHub Action will detect the SSH_DEPLOY_KEY secret and use the private key to push to the destination directory. The GitHub Action will detect the SSH_DEPLOY_KEY secret and use the private key to push to the destination directory.
Please read the [Troubleshooting](#troubleshooting) section if you encounter problems, to find how to debug possible problems. Please read the [Troubleshooting](#troubleshooting) section if you encounter problems.
### Set up using personal access token ### Set up using Personal Access Token
This does not need to be done if you chose to set up the deploy keys using the steps above. This method is here for compatibility with the initial approach of this GitHub Action. The personal access token would have access to all your repositories, so if it were to be leaked, the damage would be greater (it would allow pushes to the same repositories that the Personal Access Token owner has access and other possible associated permissions). On the other hand, the setup is a bit easier because it does not involve creating the deploy key. This does not need to be done if you chose to set up the deploy keys using the steps above. This method is here for compatibility with the initial approach of this GitHub Action. The Personal Access Token would have access to all your repositories, so if it were to be leaked, the damage would be greater: it would allow pushes to the same repositories to which the Personal Access Token owner has access, and other possible associated permissions. On the other hand, the setup is a bit easier because it does not involve creating the deploy key.
Generate your personal token following the steps: Generate your Personal Access Token using the following steps:
* Go to the general GitHub Settings (on the right-hand side on the profile picture) * Go to the general GitHub Settings (on the right-hand side on the profile picture)
![Screenshot-settings](docs/screenshots/pat-10.png) ![Screenshot-settings](docs/screenshots/pat-10.png)
@ -187,7 +191,7 @@ Generate your personal token following the steps:
![Screenshot-personal-access-token](docs/screenshots/pat-30.png) ![Screenshot-personal-access-token](docs/screenshots/pat-30.png)
* Generate a new token entering a name, expiration date and choose "Repo". Click the bottom button "Generate token". If you choose an expiration date you will need to create a new token after this date. I've chosen in this example "No expiration"; this should be used carefully. * Generate a new token entering a name and expiration date, and choose "Repo". Click the bottom button "Generate token". If you choose an expiration date you will need to create a new token after this date. In this example, I have used "No expiration"; this should be used carefully.
![Screenshot-personal-access-token-comment-expiration-scope](docs/screenshots/pat-40.png) ![Screenshot-personal-access-token-comment-expiration-scope](docs/screenshots/pat-40.png)
@ -211,7 +215,10 @@ Then make the token available to the GitHub Action using the following steps:
![Screenshot-new-repository-secret](docs/screenshots/pat-70.png) ![Screenshot-new-repository-secret](docs/screenshots/pat-70.png)
#### Example usage #### Example set up
This example shows both set up methods. Only one of the lines, `SSH_DEPLOY_KEY` or `API_TOKEN_GITHUB`, is needed according to the set up method that is being used. Do not include the other line.
```yaml ```yaml
- name: Pushes to another repository - name: Pushes to another repository
uses: cpina/github-action-push-to-another-repository@main uses: cpina/github-action-push-to-another-repository@main
@ -225,7 +232,6 @@ Then make the token available to the GitHub Action using the following steps:
user-email: carles3@pina.cat user-email: carles3@pina.cat
target-branch: main target-branch: main
``` ```
(you only need `SSH_DEPLOY_KEY` or `API_TOKEN_GITHUB` depending on the method that you used)
Working example: Working example:
@ -241,44 +247,23 @@ https://github.com/cpina/push-to-another-repository-output
### Cannot push to the destination repository ### Cannot push to the destination repository
This is the most common problem. Carefully read the logs on the GitHub Action pages of the source repositories. The action tries to write the errors and gives possible solutions / hints. The first suggestion is to follow the steps above to create the ssh keys or Personal Access Token, and set it up again, but some problems that have occurred in the past are: This is the most common problem. Carefully read the logs on the GitHub Action pages of the source repositories. The action tries to write the errors and gives possible solutions / hints. To solve this problem, the first suggestion is to follow the steps above to redo the setup following the steps above. If this still does not work, the following problems have occurred in the past, so they may be useful points to start solving the problem:
* User was logged in to GitHub with an account that did not have permission to push into the destination repository. The action could not push (permission denied). * User was logged in to GitHub with an account that did not have permission to push into the destination repository. The action could not push (permission denied).
* Token expired at some point * Expired Token.
* ssh generated key pair (public/private) was set in the wrong way. * SSH generated key pair (public/private) was set in the wrong way.
* Wrong copy-paste including spaces or other characters that did not belong to the key / token * Wrong copy-paste including spaces or other characters that did not belong to the key / token
### How to test the Personal Access Token? ### Testing the SSH keys
Test that the Personal Access Token has the correct permissions to push to the destination repository. This can be useful to check that the Personal Access Token has not expired or has the correct permissions.
In your terminal do:
```
$ git clone https://YOUR_USERNAME:YOUR_ACCESS_TOKEN@github.com/SOMETHING/REPO
# YOUR_ACCESS_TOKEN probably starts with ghp_
$ cd REPO
$ git checkout -b test
$ git push origin test
```
For example, if the token did not have permission to write (because `repo` was not enabled for this token), you might see the following:
```
$ git push origin test
remote: Permission to cpina/qdacco.git denied to cpina.
fatal: unable to access 'https://github.com/cpina/qdacco/': The requested URL returned error: 403
```
### How to test the ssh keys?
#### Test connection to GitHub using the key pair #### Test connection to GitHub using the key pair
The public key should have been added into the repository (as an SSH deploy key following the documentation above). The public key should have been added into the repository (as an SSH deploy key following the documentation above).
Once done, on your computer: Then, do:
``` ```
$ ssh -o IdentitiesOnly=yes -i .ssh/id_push-to-another-repository-output_ed25519 git@github.com $ ssh -o IdentitiesOnly=yes -i .ssh/id_push-to-another-repository-output_ed25519 git@github.com
``` ```
@ -308,46 +293,44 @@ The output will be something like:
This should match what you see in the deploy key of the destination repository. If it does not match, the deploy key is not properly set (perhaps different public/private keys were used, for example). This should match what you see in the deploy key of the destination repository. If it does not match, the deploy key is not properly set (perhaps different public/private keys were used, for example).
##### Permission of the key #### Permission of the key
Visualise the key in the destination repository, and it should say: Visualise the key in the destination repository, and it should say:
``` ```
Read/Write Read/Write
``` ```
### Testing the Personal Access Token
## FAQ If the Personal Access Token method was used for the setup, check it has the correct permissions to push to the destination repository and has not expired.
### How can I copy the whole repository? In your terminal do:
Use `source_directory: .`. This will copy all the source GitHub repository to the destination. ```
$ git clone https://YOUR_USERNAME:YOUR_ACCESS_TOKEN@github.com/SOMETHING/REPO
# YOUR_ACCESS_TOKEN probably starts with ghp_
$ cd REPO
$ git checkout -b test
$ git push origin test
```
:warning: this will also copy `.github/workflows/`! The destination repository will try to execute the action. It will probably fail because the Personal Acccess Token / ssh keys will not be available for the destination repository. For example, if the token did not have permission to write (because `repo` was not enabled for this token), you might see the following:
```
A suggestion if you are using `source_directory: .` is to disable the GitHub Actions in the destination repository. To disable Actions in the destination repository: $ git push origin test
remote: Permission to cpina/qdacco.git denied to cpina.
* Go to the destination repository (e.g. https://github.com/cpina/push-to-another-repository-output/settings) fatal: unable to access 'https://github.com/cpina/qdacco/': The requested URL returned error: 403
* Click on "Settings" ```
* On the left side bar click on "Actions" and then "General"
* Click on "Disable actions"
* Click on "Save"
### How can I copy only some files / only some directories / exclude some files / etc.?
If you need to filter files or directories (exclude files, include only some, etc.) use a step before the action to filter from a temporary directory to the directory that will be pushed. This can be done with "rsync" or any tool that you feel comfortable (could be "rclone" or "find + exec + rm", "find + exec + mv", etc. depending on your needs).
For example see the step: https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/.github/workflows/ci.yml#L21
In the step above it installs rsync (depending on your environment it will already be installed) and it copies everything from "output_temp" to "output" excluding "main.epub". Be careful with the usage of "/" in rsync directories or exclude parameter. Refer to the rsync documentation for other options such as filter using an extension, include only some files / directories, etc.
### Error: URL using bad/illegal format or missing URL ### Error: URL using bad/illegal format or missing URL
It seems that GitHub created a Personal Token with a space character. This would happen if the Personal Token had a `#` or other characters. It seems that GitHub created a Personal Access Token with a space character. This would happen if the Personal Token had a `#` or some other characters.
Please check that no space was copied by mistake to the `API_TOKEN_GITHUB`. If GitHub created a token with `#` or a space inside, I suggest creating a new one. Firstly, check that no space was copied by mistake to the `API_TOKEN_GITHUB`. If this was the case, do the correct the copy.
The problem, for those that are curious, is that the Personal Access Token is used in the git URLs and the URLs don't support `#`. The error "URL using bad/illegal format" is coming from the curl library used by git. Secondly, if GitHub created a token with `#` or a space inside, I suggest creating a new one.
Complete information: [issue 70](https://github.com/cpina/github-action-push-to-another-repository/issues/70) The problem, for those that are curious, is that the Personal Access Token is used in the git URLs and the URLs don't support `#`. The error "URL using bad/illegal format" comes from the curl library used by git.
More information: [issue 70](https://github.com/cpina/github-action-push-to-another-repository/issues/70)
### Error: remote: Repository not found. ### Error: remote: Repository not found.
@ -359,3 +342,26 @@ fatal: repository 'https://github.com/ORGNAME/REPONAME.git/' not found
See the possible solution in the comments of the [issue 75](https://github.com/cpina/github-action-push-to-another-repository/issues/75) See the possible solution in the comments of the [issue 75](https://github.com/cpina/github-action-push-to-another-repository/issues/75)
## FAQ
### How can I copy the whole repository?
Use `source_directory: .` to copy all the source GitHub repository to the destination.
:warning: this will also copy `.github/workflows/`! The destination repository will try to execute the Action. It will probably fail because the SSH keys / Personal Acccess Token will not be available for the destination repository.
A suggestion if you are using `source_directory: .` is to disable the GitHub Actions in the destination repository. To disable Actions in the destination repository:
* Go to the destination repository (e.g. https://github.com/cpina/push-to-another-repository-output/settings)
* Click on "Settings"
* On the left side bar click on "Actions" and then "General"
* Click on "Disable actions"
* Click on "Save"
### How can I copy only some files / only some directories / exclude some files / etc.?
If you need to filter files or directories (exclude files, include only some, etc.) use a step before the Action to filter these into a temporary directory that will be pushed. This can be done with "rsync" or any tool that you feel comfortable with (could be "rclone" or "find + exec + rm", "find + exec + mv", etc. depending on your needs).
For example, see the step: https://github.com/cpina/push-to-another-repository-deploy-keys-example/blob/main/.github/workflows/ci.yml#L21
In the step above, rsync is installed (depending on your environment it will already be installed) and everything is copied from "output_temp" to "output" excluding "main.epub". Be careful with the usage of "/" and the exclude parameter in rsync directories. Refer to the rsync documentation for other options such as filter using an extension to include only some files / directories, etc.