使用 dockerfile 克隆私有 git 仓库
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/23391839/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Clone private git repo with dockerfile
提问by crooksey
I have copied this code from what seems to be various working dockerfiles around, here is mine:
我已经从似乎是各种工作的 dockerfiles 复制了这段代码,这是我的:
FROM ubuntu
MAINTAINER Luke Crooks "[email protected]"
# Update aptitude with new repo
RUN apt-get update
# Install software
RUN apt-get install -y git python-virtualenv
# Make ssh dir
RUN mkdir /root/.ssh/
# Copy over private key, and set permissions
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN chown -R root:root /root/.ssh
# Create known_hosts
RUN touch /root/.ssh/known_hosts
# Remove host checking
RUN echo "Host bitbucket.org\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
# Clone the conf files into the docker container
RUN git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf
This gives me the error
这给了我错误
Step 10 : RUN git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf
---> Running in 0d244d812a54
Cloning into '/home/docker-conf'...
Warning: Permanently added 'bitbucket.org,131.103.20.167' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
2014/04/30 16:07:28 The command [/bin/sh -c git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf] returned a non-zero code: 128
This is my first time using dockerfiles, but from what I have read (and taken from working configs) I cannot see why this doesn't work.
这是我第一次使用 dockerfiles,但从我读过的(和从工作配置中获取的)我看不出为什么这不起作用。
My id_rsa is in the same folder as my dockerfile and is a copy of my local key which can clone this repo no problem.
我的 id_rsa 与我的 dockerfile 位于同一文件夹中,并且是我本地密钥的副本,它可以毫无问题地克隆这个 repo。
Edit:
编辑:
In my dockerfile I can add:
在我的 dockerfile 中,我可以添加:
RUN cat /root/.ssh/id_rsa
And it prints out the correct key, so I know its being copied correctly.
它打印出正确的密钥,所以我知道它被正确复制。
I have also tried to do as noah advised and ran:
我也尝试按照诺亚的建议去做并运行:
RUN echo "Host bitbucket.org\n\tIdentityFile /root/.ssh/id_rsa\n\tStrictHostKeyChecking no" >> /etc/ssh/ssh_config
This sadly also doesn't work.
可悲的是,这也不起作用。
回答by crooksey
My key was password protected which was causing the problem, a working file is now listed below (for help of future googlers)
我的密钥受密码保护,这导致了问题,下面列出了一个工作文件(以帮助未来的谷歌员工)
FROM ubuntu
MAINTAINER Luke Crooks "[email protected]"
# Update aptitude with new repo
RUN apt-get update
# Install software
RUN apt-get install -y git
# Make ssh dir
RUN mkdir /root/.ssh/
# Copy over private key, and set permissions
# Warning! Anyone who gets their hands on this image will be able
# to retrieve this private key file from the corresponding image layer
ADD id_rsa /root/.ssh/id_rsa
# Create known_hosts
RUN touch /root/.ssh/known_hosts
# Add bitbuckets key
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git
回答by Marcin R
You should create new SSH key set for that Docker image, as you probably don't want to embed there your own private key. To make it work, you'll have to add that key to deployment keys in your git repository. Here's complete recipe:
您应该为该 Docker 映像创建新的 SSH 密钥集,因为您可能不想在其中嵌入您自己的私钥。要使其工作,您必须将该密钥添加到 git 存储库中的部署密钥。这是完整的食谱:
Generate ssh keys with
ssh-keygen -q -t rsa -N '' -f repo-key
which will give you repo-key and repo-key.pub files.Add repo-key.pub to your repository deployment keys.
On GitHub, go to [your repository] -> Settings -> Deploy keysAdd something like this to your Dockerfile:
ADD repo-key / RUN \ chmod 600 /repo-key && \ echo "IdentityFile /repo-key" >> /etc/ssh/ssh_config && \ echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \ // your git clone commands here...
生成 ssh 密钥
ssh-keygen -q -t rsa -N '' -f repo-key
,它将为您提供 repo-key 和 repo-key.pub 文件。将 repo-key.pub 添加到您的存储库部署密钥。
在 GitHub 上,转到 [您的存储库] -> 设置 -> 部署密钥将这样的内容添加到您的 Dockerfile 中:
ADD repo-key / RUN \ chmod 600 /repo-key && \ echo "IdentityFile /repo-key" >> /etc/ssh/ssh_config && \ echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \ // your git clone commands here...
Note that above switches off StrictHostKeyChecking, so you don't need .ssh/known_hosts. Although I probably like more the solution with ssh-keyscan in one of the answers above.
请注意,上面关闭了 StrictHostKeyChecking,因此您不需要 .ssh/known_hosts。尽管我可能更喜欢上述答案之一中的 ssh-keyscan 解决方案。
回答by Calvin Froedge
There's no need to fiddle around with ssh configurations. Use a configuration file (not a Dockerfile) that contains environment variables, and have a shell script update your docker file at runtime. You keep tokens out of your Dockerfiles and you can clone over https (no need to generate or pass around ssh keys).
没有必要摆弄 ssh 配置。使用包含环境变量的配置文件(不是 Dockerfile),并让 shell 脚本在运行时更新您的 docker 文件。您将令牌保留在 Dockerfile 之外,并且可以通过 https 进行克隆(无需生成或传递 ssh 密钥)。
Go to Settings > Personal Access Tokens
- Generate a personal access token with
repo
scope enabled. - Clone like this:
git clone https://[email protected]/user-or-org/repo
- 生成
repo
启用范围的个人访问令牌。 - 像这样克隆:
git clone https://[email protected]/user-or-org/repo
Some commenters have noted that if you use a shared Dockerfile, this could expose your access key to other people on your project. While this may or may not be a concern for your specific use case, here are some ways you can deal with that:
一些评论者指出,如果您使用共享 Dockerfile,这可能会将您的访问密钥暴露给您项目中的其他人。虽然这可能是也可能不是您的特定用例的问题,但您可以通过以下一些方法来解决这个问题:
- Use a shell script to accept arguments which could contain your key as a variable. Replace a variable in your Dockerfile with
sed
or similar, i.e. calling the script withsh rundocker.sh MYTOKEN=foo
which would replace onhttps://{{MY_TOKEN}}@github.com/user-or-org/repo
. Note that you could also use a configuration file (in .yml or whatever format you want) to do the same thing but with environment variables. - Create a github user (and generate an access token for) for that project only
- 使用 shell 脚本接受可能包含您的密钥作为变量的参数。将 Dockerfile 中的变量替换为
sed
或类似的,即调用sh rundocker.sh MYTOKEN=foo
将替换 on的脚本https://{{MY_TOKEN}}@github.com/user-or-org/repo
。请注意,您还可以使用配置文件(.yml 或您想要的任何格式)来执行相同的操作,但使用环境变量。 - 仅为该项目创建一个 github 用户(并为其生成访问令牌)
回答by jaker
Another option is to use a multi-stage docker build to ensure that your SSH keys are not included in the final image.
另一种选择是使用多阶段 docker 构建来确保您的 SSH 密钥不包含在最终映像中。
As described in my postyou can prepare your intermediate image with the required dependencies to git clone and then COPY
the required files into your final image.
如我的帖子中所述,您可以准备具有所需依赖项的中间映像以 git clone,然后COPY
将所需文件放入最终映像中。
Additionally if we LABEL
our intermediate layers, we can even delete them from the machine when finished.
此外,如果我们是LABEL
中间层,我们甚至可以在完成后从机器中删除它们。
# Choose and name our temporary image.
FROM alpine as intermediate
# Add metadata identifying these images as our build containers (this will be useful later!)
LABEL stage=intermediate
# Take an SSH key as a build argument.
ARG SSH_KEY
# Install dependencies required to git clone.
RUN apk update && \
apk add --update git && \
apk add --update openssh
# 1. Create the SSH directory.
# 2. Populate the private key file.
# 3. Set the required permissions.
# 4. Add github to our list of known hosts for ssh.
RUN mkdir -p /root/.ssh/ && \
echo "$SSH_KEY" > /root/.ssh/id_rsa && \
chmod -R 600 /root/.ssh/ && \
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
# Clone a repository (my website in this case)
RUN git clone [email protected]:janakerman/janakerman.git
# Choose the base image for our final image
FROM alpine
# Copy across the files from our `intermediate` container
RUN mkdir files
COPY --from=intermediate /janakerman/README.md /files/README.md
We can then build:
然后我们可以构建:
MY_KEY=$(cat ~/.ssh/id_rsa)
docker build --build-arg SSH_KEY="$MY_KEY" --tag clone-example .
Prove our SSH keys are gone:
证明我们的 SSH 密钥不见了:
docker run -ti --rm clone-example cat /root/.ssh/id_rsa
Clean intermediate images from the build machine:
从构建机器清理中间图像:
docker rmi -f $(docker images -q --filter label=stage=intermediate)
回答by Nomce
For bitbucket repository, generate App Password (Bitbucket settings -> Access Management -> App Password, see the image) with read access to the repo and project.
对于 bitbucket 存储库,生成具有对 repo 和项目的读取权限的应用密码(Bitbucket 设置 -> 访问管理 -> 应用密码,请参见图像)。
Then the command that you should use is:
那么你应该使用的命令是:
git clone https://username:[email protected]/reponame/projectname.git
回答by BMitch
You often do not want to perform a git clone
of a private repo from within the docker build. Doing the clone there involves placing the private ssh credentials inside the image where they can be later extracted by anyone with access to your image.
您通常不想git clone
从 docker 构建中执行私有存储库。在那里进行克隆涉及将私有 ssh 凭据放置在图像中,以后任何有权访问您的图像的人都可以提取这些凭据。
Instead, the common practice is to clone the git repo from outside of docker in your CI tool of choice, and simply COPY
the files into the image. This has a second benefit: docker caching. Docker caching looks at the command being run, environment variables it includes, input files, etc, and if they are identical to a previous build from the same parent step, it reuses that previous cache. With a git clone
command, the command itself is identical, so docker will reuse the cache even if the external git repo is changed. However, a COPY
command will look at the files in the build context and can see if they are identical or have been updated, and use the cache only when it's appropriate.
相反,通常的做法是在您选择的 CI 工具中从 docker 外部克隆 git repo,并将COPY
文件简单地复制到映像中。这还有第二个好处:docker 缓存。Docker 缓存查看正在运行的命令、它包含的环境变量、输入文件等,如果它们与来自同一父步骤的先前构建相同,则它会重用先前的缓存。对于git clone
命令,命令本身是相同的,因此即使外部 git repo 更改,docker 也会重用缓存。但是,COPY
命令将查看构建上下文中的文件,并可以查看它们是否相同或已更新,并且仅在适当时使用缓存。
If you are going to add credentials into your build, consider doing so with a multi-stage build, and only placing those credentials in an early stage that is never tagged and pushed outside of your build host. The result looks like:
如果您要将凭据添加到构建中,请考虑使用多阶段构建,并且仅将这些凭据放置在永远不会标记并推送到构建主机之外的早期阶段。结果如下:
FROM ubuntu as clone
# Update aptitude with new repo
RUN apt-get update \
&& apt-get install -y git
# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
# Copy over private key, and set permissions
# Warning! Anyone who gets their hands on this image will be able
# to retrieve this private key file from the corresponding image layer
COPY id_rsa /root/.ssh/id_rsa
# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git
FROM ubuntu as release
LABEL maintainer="Luke Crooks <[email protected]>"
COPY --from=clone /repo /repo
...
More recently, BuildKit has been testing some experimental features that allow you to pass an ssh key in as a mount that never gets written to the image:
最近,BuildKit 一直在测试一些实验性功能,这些功能允许您将 ssh 密钥作为永远不会写入图像的挂载传递:
# syntax=docker/dockerfile:experimental
FROM ubuntu as clone
LABEL maintainer="Luke Crooks <[email protected]>"
# Update aptitude with new repo
RUN apt-get update \
&& apt-get install -y git
# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
# Clone the conf files into the docker container
RUN --mount=type=secret,id=ssh_id,target=/root/.ssh/id_rsa \
git clone [email protected]:User/repo.git
And you can build that with:
您可以使用以下方法构建它:
$ DOCKER_BUILDKIT=1 docker build -t your_image_name \
--secret id=ssh_id,src=$(pwd)/id_rsa .
Note that this still requires your ssh key to not be password protected, but you can at least run the build in a single stage, removing a COPY command, and avoiding the ssh credential from ever being part of an image.
请注意,这仍然要求您的 ssh 密钥不受密码保护,但您至少可以在单个阶段运行构建,删除 COPY 命令,并避免 ssh 凭据成为映像的一部分。
BuildKit also added a feature just for ssh which allows you to still have your password protected ssh keys, the result looks like:
BuildKit 还为 ssh 添加了一项功能,它允许您仍然拥有受密码保护的 ssh 密钥,结果如下所示:
# syntax=docker/dockerfile:experimental
FROM ubuntu as clone
LABEL maintainer="Luke Crooks <[email protected]>"
# Update aptitude with new repo
RUN apt-get update \
&& apt-get install -y git
# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
&& touch /root/.ssh/known_hosts \
&& ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts
# Clone the conf files into the docker container
RUN --mount=type=ssh \
git clone [email protected]:User/repo.git
And you can build that with:
您可以使用以下方法构建它:
$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_rsa
(Input your passphrase here)
$ DOCKER_BUILDKIT=1 docker build -t your_image_name \
--ssh default=$SSH_AUTH_SOCK .
Again, this is injected into the build without ever being written to an image layer, removing the risk that the credential could accidentally leak out.
同样,这被注入到构建中,而不会被写入图像层,从而消除了凭据可能意外泄漏的风险。
To force docker to run the git clone
even when the lines before have been cached, you can inject a build ARG that changes with each build to break the cache. That looks like:
要强制 dockergit clone
在之前的行已被缓存时运行,您可以注入一个构建 ARG,该 ARG 随每次构建而变化以破坏缓存。看起来像:
# inject a datestamp arg which is treated as an environment variable and
# will break the cache for the next RUN command
ARG DATE_STAMP
# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git
Then you inject that changing arg in the docker build command:
然后你在 docker build 命令中注入改变的 arg:
date_stamp=$(date +%Y%m%d-%H%M%S)
docker build --build-arg DATE_STAMP=$date_stamp .
回答by tvgriek
Above solutions did not work for bitbucket. I figured this does the trick:
上述解决方案不适用于 bitbucket。我认为这可以解决问题:
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts \
&& eval `ssh-agent` \
&& ssh-add ~/.ssh/[key] \
&& git clone [email protected]:[team]/[repo].git