使用 dockerfile 克隆私有 git 仓库

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/23391839/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-10 18:03:31  来源:igfitidea点击:

Clone private git repo with dockerfile

gitbitbucketdocker

提问by crooksey

I have copied this code from what seems to be various working dockerfiles around, here is mine:

我已经从似乎是各种工作的 dockerfiles 复制了这段代码,这是我的:

FROM ubuntu

MAINTAINER Luke Crooks "[email protected]"

# Update aptitude with new repo
RUN apt-get update

# Install software 
RUN apt-get install -y git python-virtualenv

# Make ssh dir
RUN mkdir /root/.ssh/

# Copy over private key, and set permissions
ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN chown -R root:root /root/.ssh

# Create known_hosts
RUN touch /root/.ssh/known_hosts

# Remove host checking
RUN echo "Host bitbucket.org\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

# Clone the conf files into the docker container
RUN git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf

This gives me the error

这给了我错误

Step 10 : RUN git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf
 ---> Running in 0d244d812a54
Cloning into '/home/docker-conf'...
Warning: Permanently added 'bitbucket.org,131.103.20.167' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
2014/04/30 16:07:28 The command [/bin/sh -c git clone [email protected]:Pumalo/docker-conf.git /home/docker-conf] returned a non-zero code: 128

This is my first time using dockerfiles, but from what I have read (and taken from working configs) I cannot see why this doesn't work.

这是我第一次使用 dockerfiles,但从我读过的(和从工作配置中获取的)我看不出为什么这不起作用。

My id_rsa is in the same folder as my dockerfile and is a copy of my local key which can clone this repo no problem.

我的 id_rsa 与我的 dockerfile 位于同一文件夹中,并且是我本地密钥的副本,它可以毫无问题地克隆这个 repo。

Edit:

编辑:

In my dockerfile I can add:

在我的 dockerfile 中,我可以添加:

RUN cat /root/.ssh/id_rsa

And it prints out the correct key, so I know its being copied correctly.

它打印出正确的密钥,所以我知道它被正确复制。

I have also tried to do as noah advised and ran:

我也尝试按照诺亚的建议去做并运行:

RUN echo "Host bitbucket.org\n\tIdentityFile /root/.ssh/id_rsa\n\tStrictHostKeyChecking no" >> /etc/ssh/ssh_config

This sadly also doesn't work.

可悲的是,这也不起作用。

回答by crooksey

My key was password protected which was causing the problem, a working file is now listed below (for help of future googlers)

我的密钥受密码保护,这导致了问题,下面列出了一个工作文件(以帮助未来的谷歌员工)

FROM ubuntu

MAINTAINER Luke Crooks "[email protected]"

# Update aptitude with new repo
RUN apt-get update

# Install software 
RUN apt-get install -y git
# Make ssh dir
RUN mkdir /root/.ssh/

# Copy over private key, and set permissions
# Warning! Anyone who gets their hands on this image will be able
# to retrieve this private key file from the corresponding image layer
ADD id_rsa /root/.ssh/id_rsa

# Create known_hosts
RUN touch /root/.ssh/known_hosts
# Add bitbuckets key
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git

回答by Marcin R

You should create new SSH key set for that Docker image, as you probably don't want to embed there your own private key. To make it work, you'll have to add that key to deployment keys in your git repository. Here's complete recipe:

您应该为该 Docker 映像创建新的 SSH 密钥集,因为您可能不想在其中嵌入您自己的私钥。要使其工作,您必须将该密钥添加到 git 存储库中的部署密钥。这是完整的食谱:

  1. Generate ssh keys with ssh-keygen -q -t rsa -N '' -f repo-keywhich will give you repo-key and repo-key.pub files.

  2. Add repo-key.pub to your repository deployment keys.
    On GitHub, go to [your repository] -> Settings -> Deploy keys

  3. Add something like this to your Dockerfile:

    ADD repo-key /
    RUN \
      chmod 600 /repo-key && \  
      echo "IdentityFile /repo-key" >> /etc/ssh/ssh_config && \  
      echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \  
      // your git clone commands here...
    
  1. 生成 ssh 密钥ssh-keygen -q -t rsa -N '' -f repo-key,它将为您提供 repo-key 和 repo-key.pub 文件。

  2. 将 repo-key.pub 添加到您的存储库部署密钥。
    在 GitHub 上,转到 [您的存储库] -> 设置 -> 部署密钥

  3. 将这样的内容添加到您的 Dockerfile 中:

    ADD repo-key /
    RUN \
      chmod 600 /repo-key && \  
      echo "IdentityFile /repo-key" >> /etc/ssh/ssh_config && \  
      echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \  
      // your git clone commands here...
    

Note that above switches off StrictHostKeyChecking, so you don't need .ssh/known_hosts. Although I probably like more the solution with ssh-keyscan in one of the answers above.

请注意,上面关闭了 StrictHostKeyChecking,因此您不需要 .ssh/known_hosts。尽管我可能更喜欢上述答案之一中的 ssh-keyscan 解决方案。

回答by Calvin Froedge

There's no need to fiddle around with ssh configurations. Use a configuration file (not a Dockerfile) that contains environment variables, and have a shell script update your docker file at runtime. You keep tokens out of your Dockerfiles and you can clone over https (no need to generate or pass around ssh keys).

没有必要摆弄 ssh 配置。使用包含环境变量的配置文件(不是 Dockerfile),并让 shell 脚本在运行时更新您的 docker 文件。您将令牌保留在 Dockerfile 之外,并且可以通过 https 进行克隆(无需生成或传递 ssh 密钥)。

Go to Settings > Personal Access Tokens

转到设置 > 个人访问令牌

  • Generate a personal access token with reposcope enabled.
  • Clone like this: git clone https://[email protected]/user-or-org/repo
  • 生成repo启用范围的个人访问令牌。
  • 像这样克隆: git clone https://[email protected]/user-or-org/repo

Some commenters have noted that if you use a shared Dockerfile, this could expose your access key to other people on your project. While this may or may not be a concern for your specific use case, here are some ways you can deal with that:

一些评论者指出,如果您使用共享 Dockerfile,这可能会将您的访问密钥暴露给您项目中的其他人。虽然这可能是也可能不是您的特定用例的问题,但您可以通过以下一些方法来解决这个问题:

  • Use a shell script to accept arguments which could contain your key as a variable. Replace a variable in your Dockerfile with sedor similar, i.e. calling the script with sh rundocker.sh MYTOKEN=foowhich would replace on https://{{MY_TOKEN}}@github.com/user-or-org/repo. Note that you could also use a configuration file (in .yml or whatever format you want) to do the same thing but with environment variables.
  • Create a github user (and generate an access token for) for that project only
  • 使用 shell 脚本接受可能包含您的密钥作为变量的参数。将 Dockerfile 中的变量替换为sed或类似的,即调用sh rundocker.sh MYTOKEN=foo将替换 on的脚本https://{{MY_TOKEN}}@github.com/user-or-org/repo。请注意,您还可以使用配置文件(.yml 或您想要的任何格式)来执行相同的操作,但使用环境变量。
  • 仅为该项目创建一个 github 用户(并为其生成访问令牌)

回答by jaker

Another option is to use a multi-stage docker build to ensure that your SSH keys are not included in the final image.

另一种选择是使用多阶段 docker 构建来确保您的 SSH 密钥不包含在最终映像中。

As described in my postyou can prepare your intermediate image with the required dependencies to git clone and then COPYthe required files into your final image.

如我的帖子中所述,您可以准备具有所需依赖项的中间映像以 git clone,然后COPY将所需文件放入最终映像中。

Additionally if we LABELour intermediate layers, we can even delete them from the machine when finished.

此外,如果我们是LABEL中间层,我们甚至可以在完成后从机器中删除它们。

# Choose and name our temporary image.
FROM alpine as intermediate
# Add metadata identifying these images as our build containers (this will be useful later!)
LABEL stage=intermediate

# Take an SSH key as a build argument.
ARG SSH_KEY

# Install dependencies required to git clone.
RUN apk update && \
    apk add --update git && \
    apk add --update openssh

# 1. Create the SSH directory.
# 2. Populate the private key file.
# 3. Set the required permissions.
# 4. Add github to our list of known hosts for ssh.
RUN mkdir -p /root/.ssh/ && \
    echo "$SSH_KEY" > /root/.ssh/id_rsa && \
    chmod -R 600 /root/.ssh/ && \
    ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts

# Clone a repository (my website in this case)
RUN git clone [email protected]:janakerman/janakerman.git

# Choose the base image for our final image
FROM alpine

# Copy across the files from our `intermediate` container
RUN mkdir files
COPY --from=intermediate /janakerman/README.md /files/README.md

We can then build:

然后我们可以构建:

MY_KEY=$(cat ~/.ssh/id_rsa)
docker build --build-arg SSH_KEY="$MY_KEY" --tag clone-example .

Prove our SSH keys are gone:

证明我们的 SSH 密钥不见了:

docker run -ti --rm clone-example cat /root/.ssh/id_rsa

Clean intermediate images from the build machine:

从构建机器清理中间图像:

docker rmi -f $(docker images -q --filter label=stage=intermediate)

回答by Nomce

For bitbucket repository, generate App Password (Bitbucket settings -> Access Management -> App Password, see the image) with read access to the repo and project.

对于 bitbucket 存储库,生成具有对 repo 和项目的读取权限的应用密码(Bitbucket 设置 -> 访问管理 -> 应用密码,请参见图像)。

bitbucket user menu

bitbucket 用户菜单

Then the command that you should use is:

那么你应该使用的命令是:

git clone https://username:[email protected]/reponame/projectname.git

回答by BMitch

You often do not want to perform a git cloneof a private repo from within the docker build. Doing the clone there involves placing the private ssh credentials inside the image where they can be later extracted by anyone with access to your image.

您通常不想git clone从 docker 构建中执行私有存储库。在那里进行克隆涉及将私有 ssh 凭据放置在图像中,以后任何有权访问您的图像的人都可以提取这些凭据。

Instead, the common practice is to clone the git repo from outside of docker in your CI tool of choice, and simply COPYthe files into the image. This has a second benefit: docker caching. Docker caching looks at the command being run, environment variables it includes, input files, etc, and if they are identical to a previous build from the same parent step, it reuses that previous cache. With a git clonecommand, the command itself is identical, so docker will reuse the cache even if the external git repo is changed. However, a COPYcommand will look at the files in the build context and can see if they are identical or have been updated, and use the cache only when it's appropriate.

相反,通常的做法是在您选择的 CI 工具中从 docker 外部克隆 git repo,并将COPY文件简单地复制到映像中。这还有第二个好处:docker 缓存。Docker 缓存查看正在运行的命令、它包含的环境变量、输入文件等,如果它们与来自同一父步骤的先前构建相同,则它会重用先前的缓存。对于git clone命令,命令本身是相同的,因此即使外部 git repo 更改,docker 也会重用缓存。但是,COPY命令将查看构建上下文中的文件,并可以查看它们是否相同或已更新,并且仅在适当时使用缓存。



If you are going to add credentials into your build, consider doing so with a multi-stage build, and only placing those credentials in an early stage that is never tagged and pushed outside of your build host. The result looks like:

如果您要将凭据添加到构建中,请考虑使用多阶段构建,并且仅将这些凭据放置在永远不会标记并推送到构建主机之外的早期阶段。结果如下:

FROM ubuntu as clone

# Update aptitude with new repo
RUN apt-get update \
 && apt-get install -y git
# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
 && touch /root/.ssh/known_hosts \
 && ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

# Copy over private key, and set permissions
# Warning! Anyone who gets their hands on this image will be able
# to retrieve this private key file from the corresponding image layer
COPY id_rsa /root/.ssh/id_rsa

# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git

FROM ubuntu as release
LABEL maintainer="Luke Crooks <[email protected]>"

COPY --from=clone /repo /repo
...


More recently, BuildKit has been testing some experimental features that allow you to pass an ssh key in as a mount that never gets written to the image:

最近,BuildKit 一直在测试一些实验性功能,这些功能允许您将 ssh 密钥作为永远不会写入图像的挂载传递:

# syntax=docker/dockerfile:experimental
FROM ubuntu as clone
LABEL maintainer="Luke Crooks <[email protected]>"

# Update aptitude with new repo
RUN apt-get update \
 && apt-get install -y git

# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
 && touch /root/.ssh/known_hosts \
 && ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

# Clone the conf files into the docker container
RUN --mount=type=secret,id=ssh_id,target=/root/.ssh/id_rsa \
    git clone [email protected]:User/repo.git

And you can build that with:

您可以使用以下方法构建它:

$ DOCKER_BUILDKIT=1 docker build -t your_image_name \
  --secret id=ssh_id,src=$(pwd)/id_rsa .

Note that this still requires your ssh key to not be password protected, but you can at least run the build in a single stage, removing a COPY command, and avoiding the ssh credential from ever being part of an image.

请注意,这仍然要求您的 ssh 密钥不受密码保护,但您至少可以在单个阶段运行构建,删除 COPY 命令,并避免 ssh 凭据成为映像的一部分。



BuildKit also added a feature just for ssh which allows you to still have your password protected ssh keys, the result looks like:

BuildKit 还为 ssh 添加了一项功能,它允许您仍然拥有受密码保护的 ssh 密钥,结果如下所示:

# syntax=docker/dockerfile:experimental
FROM ubuntu as clone
LABEL maintainer="Luke Crooks <[email protected]>"

# Update aptitude with new repo
RUN apt-get update \
 && apt-get install -y git

# Make ssh dir
# Create known_hosts
# Add bitbuckets key
RUN mkdir /root/.ssh/ \
 && touch /root/.ssh/known_hosts \
 && ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

# Clone the conf files into the docker container
RUN --mount=type=ssh \
    git clone [email protected]:User/repo.git

And you can build that with:

您可以使用以下方法构建它:

$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_rsa
(Input your passphrase here)
$ DOCKER_BUILDKIT=1 docker build -t your_image_name \
  --ssh default=$SSH_AUTH_SOCK .

Again, this is injected into the build without ever being written to an image layer, removing the risk that the credential could accidentally leak out.

同样,这被注入到构建中,而不会被写入图像层,从而消除了凭据可能意外泄漏的风险。



To force docker to run the git cloneeven when the lines before have been cached, you can inject a build ARG that changes with each build to break the cache. That looks like:

要强制 dockergit clone在之前的行已被缓存时运行,您可以注入一个构建 ARG,该 ARG 随每次构建而变化以破坏缓存。看起来像:

# inject a datestamp arg which is treated as an environment variable and
# will break the cache for the next RUN command
ARG DATE_STAMP
# Clone the conf files into the docker container
RUN git clone [email protected]:User/repo.git

Then you inject that changing arg in the docker build command:

然后你在 docker build 命令中注入改变的 arg:

date_stamp=$(date +%Y%m%d-%H%M%S)
docker build --build-arg DATE_STAMP=$date_stamp .

回答by tvgriek

Above solutions did not work for bitbucket. I figured this does the trick:

上述解决方案不适用于 bitbucket。我认为这可以解决问题:

RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts \
    && eval `ssh-agent` \
    && ssh-add ~/.ssh/[key] \
    && git clone [email protected]:[team]/[repo].git