Java Docker 缓存 gradle 依赖项

Docker cache gradle dependencies

提问by martin treurnicht

I'm trying to deploy our java web application to aws elastic beanstalk using docker, the idea is to be able to run the container locally for development and testing and eventually push it up to production using git.

我正在尝试使用 docker 将我们的 java web 应用程序部署到 aws elastic beanstalk，这个想法是能够在本地运行容器进行开发和测试，并最终使用 git 将其推送到生产环境。

I've created a base image that has tomcat8 and java8 installed, the image that performs the gradle builds inherit from this base image, speeding up build process.

我创建了一个安装了 tomcat8 和 java8 的基础镜像，执行 gradle 构建的镜像继承自这个基础镜像，加快了构建过程。

All works well, except for the fact that the inheriting application container that gets built using docker doesn't seem to cache the gradle dependencies, it downloads it every time, including gradlew. We build our web application using the following command:

一切正常，除了使用 docker 构建的继承应用程序容器似乎没有缓存 gradle 依赖项之外，它每次都下载它，包括 gradlew。我们使用以下命令构建我们的 Web 应用程序：

./gradlew war

./gradlew war

Is there some way that i can cache the files in ~/.gradlethis would speed my build up dramatically.

有什么方法可以缓存文件，~/.gradle这会大大加快我的构建速度。

This isn't so much of an issue on beanstalk but is a big problem for devs trying to build and run locally as this does take a lot of time, as you can imagine.

这在 beanstalk 上不是什么大问题，但对于尝试在本地构建和运行的开发人员来说是一个大问题，因为这确实需要很多时间，正如您想象的那样。

The base image dockerfile:

基础镜像 dockerfile：

FROM phusion/baseimage
EXPOSE 8080
RUN apt-get update
RUN add-apt-repository ppa:webupd8team/java
RUN apt-get update
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
RUN apt-get -y install oracle-java8-installer
RUN java -version
ENV TOMCAT_VERSION 8.0.9
RUN wget --quiet --no-cookies http://archive.apache.org/dist/tomcat/tomcat-8/v${TOMCAT_VERSION}/bin/apache-tomcat-${TOMCAT_VERSION}.tar.gz -O /tmp/catalina.tar.gz
# Unpack
RUN tar xzf /tmp/catalina.tar.gz -C /opt
RUN mv /opt/apache-tomcat-${TOMCAT_VERSION} /opt/tomcat
RUN ln -s /opt/tomcat/logs /var/log/tomcat
RUN rm /tmp/catalina.tar.gz
# Remove unneeded apps
RUN rm -rf /opt/tomcat/webapps/examples
RUN rm -rf /opt/tomcat/webapps/docs
RUN rm -rf /opt/tomcat/webapps/ROOT
ENV CATALINA_HOME /opt/tomcat
ENV PATH $PATH:$CATALINA_HOME/bin
ENV CATALINA_OPTS $PARAM1
# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]

The application dockerfile:

应用程序dockerfile：

FROM <tag name here for base image>
RUN mkdir ~/.gradle
# run some extra stuff here to add things to gradle.properties file
# Add project Source
ADD . /var/app/myapp
# Compile and Deploy Application, this is what is downloading gradlew and all the maven dependencies every time, if only there was a way to take the changes it makes to ~/.gradle and persist it as a cache layer
RUN cd /var/app/myapp/ && ./gradlew war
RUN mv /var/app/myapp/build/libs/myapp.war /opt/tomcat/webapps/ROOT.war
# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]

回答by gesellix

You might want to consider splitting your application image to two images: one for building the myapp.war and the other for running your application. That way, you can use docker volumes during the actual build and bind the host's ~/.gradlefolder into the container performing the build. Instead of only one step to run your application, you would have more steps, though. Example:

您可能需要考虑将应用程序映像拆分为两个映像：一个用于构建 myapp.war，另一个用于运行您的应用程序。这样，您可以在实际构建期间使用 docker 卷并将主机的~/.gradle文件夹绑定到执行构建的容器中。但是，您将有更多的步骤来运行您的应用程序，而不是只有一个步骤。例子：

builder image

建设者形象

FROM <tag name here for base image including all build time dependencies>

# Add project Source
# -> you can use a project specific gradle.properties in your project root
# in order to override global/user gradle.properties
ADD . /var/app/myapp

RUN mkdir -p /root/.gradle
ENV HOME /root
# declare shared volume path
VOLUME /root/.gradle
WORKDIR /var/app/myapp/ 

# Compile only
CMD ["./gradlew", "war"]

application image

应用图片

FROM <tag name here for application base image>

ADD ./ROOT.war /opt/tomcat/webapps/ROOT.war

# Start Tomcat
CMD ["/opt/tomcat/bin/catalina.sh", "run"]

How to use in your project root, assuming the builder Dockerfile is located there and the application Dockerfile is located at the webappsubfolder (or any other path you prefer):

如何在您的项目根目录中使用，假设构建器 Dockerfile 位于那里并且应用程序 Dockerfile 位于webapp子文件夹（或您喜欢的任何其他路径）：

$ docker build -t builder .
$ docker run --name=build-result -v ~/.gradle/:/root/.gradle/ builder
$ docker cp build-result:/var/app/myapp/myapp.war webapp/ROOT.war
$ cd webapp
$ docker build -t application .
$ docker run -d -P application

I haven't tested the shown code, but I hope you get the idea. The example might even be improved by using data volumes for the .gradle/ cache, see the Docker user guidefor details.

我还没有测试显示的代码，但我希望你能明白。该示例甚至可以通过将数据卷用于 .gradle/ 缓存来改进，有关详细信息，请参阅 Docker用户指南。

回答by Daniel Wei

I

一世

Add resolveDependenciestask in build.gradle:

在 build.gradle 中添加resolveDependencies任务：

task resolveDependencies {
    doLast {
        project.rootProject.allprojects.each { subProject ->
            subProject.buildscript.configurations.each { configuration ->
                configuration.resolve()
            }
            subProject.configurations.each { configuration ->
                configuration.resolve()
            }
        }
    }
}

and update Dockerfile:

并更新 Dockerfile：

ADD build.gradle /opt/app/
WORKDIR /opt/app
RUN gradle resolveDependencies

ADD . .

RUN gradle build -x test --parallel && \
    touch build/libs/api.jar

---

---

II

二

Bellow is what I do now:

波纹管是我现在所做的：

build.gradle

构建.gradle

ext {
    speed = project.hasProperty('speed') ? project.getProperty('speed') : false
    offlineCompile = new File("$buildDir/output/lib")
}

dependencies {
    if (speed) {
        compile fileTree(dir: offlineCompile, include: '*.jar')
    } else {
        // ...dependencies
    }
}

task downloadRepos(type: Copy) {
    from configurations.all
    into offlineCompile
}

Dockerfile

文件

ADD build.gradle /opt/app/
WORKDIR /opt/app

RUN gradle downloadRepos

ADD . /opt/app
RUN gradle build -Pspeed=true

回答by Tibor Koch

try changing the gradle user home directory

尝试更改 gradle 用户主目录

RUN mkdir -p /opt/gradle/.gradle
ENV GRADLE_USER_HOME=/opt/gradle/.gradle

运行 mkdir -p /opt/gradle/.gradle
ENV GRADLE_USER_HOME=/opt/gradle/.gradle

回答by Sairam Krish

I faced this issue. As you might agree, it is a best practice to download dependencies alone as a separate step while building the docker image. It becomes little tricky with gradle, since there is no direct support for downloading just dependencies.

我遇到了这个问题。您可能同意，最佳做法是在构建 docker 映像时单独下载依赖项作为单独的步骤。使用 gradle 变得有点棘手，因为没有直接支持下载依赖项。

Option 1 : Using docker-gradle Docker image

选项 1：使用 docker-gradle Docker 镜像

---

---

We can use pre-built gradle docker image to build the application. This ensures that it's not a local system build but a build done on a clean docker image.

我们可以使用预先构建的 gradle docker 镜像来构建应用程序。这确保它不是本地系统构建，而是在干净的 docker 映像上完成的构建。

docker volume create --name gradle-cache
docker run --rm -v gradle-cache:/home/gradle/.gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:4.7.0-jdk8-alpine gradle build
ls -ltrh ./build/libs

• gradle cache is loaded here as a volume. So subsequent builds will reuse the downloaded dependencies.
• After this, we could have a Dockerfile to take this artifact and generate application specific image to run the application.
• This way, the builder image is not required. Application build flow and Application run flow is separated out.
• Since the gradle-cache volume is mounted, we could reuse the downloaded dependencies across different gradle projects.

• gradle 缓存在此处作为卷加载。因此后续构建将重用下载的依赖项。
• 在此之后，我们可以有一个 Dockerfile 来获取这个工件并生成应用程序特定的图像来运行应用程序。
• 这样，不需要构建器映像。应用程序构建流程和应用程序运行流程是分开的。
• 由于 gradle-cache 卷已挂载，我们可以在不同的 gradle 项目中重用下载的依赖项。

Option 2 : Multi-stage build

选项 2：多阶段构建

---

---

----- Dockerfile -----

----- Dockerfile -----

FROM openjdk:8 AS TEMP_BUILD_IMAGE
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY build.gradle settings.gradle gradlew $APP_HOME
COPY gradle $APP_HOME/gradle
RUN ./gradlew build || return 0 
COPY . .
RUN ./gradlew build

FROM openjdk:8
ENV ARTIFACT_NAME=your-application.jar
ENV APP_HOME=/usr/app/
WORKDIR $APP_HOME
COPY --from=TEMP_BUILD_IMAGE $APP_HOME/build/libs/$ARTIFACT_NAME .
EXPOSE 8080
CMD ["java","-jar",$ARTIFACT_NAME]

In the above Dockerfile

在上面的Dockerfile中

• First we try to copy the project's gradle files alone, like build.gradle, gradlew etc.,
• Then we copy the gradle directory itself
• And then we try to run the build. At this point, there is no other source code files exists in the directory. So build will fail. But before that it will download the dependencies.?
• Since we expect the build to fail, I have tried a simple technique to return 0 and allow the docker to continue execution
• this will speed up the subsequent build flows, since all the dependencies are downloaded and docker cached this layer. Comparatively, Volume mounting the gradle cache directory is still the best approach.
• The above example also showcases multi-stage docker imagebuilding, which avoid multiple docker build files.

• 首先我们尝试单独复制项目的gradle文件，比如build.gradle、gradlew等，
• 然后我们复制gradle目录本身
• 然后我们尝试运行构建。此时，目录中不存在其他源代码文件。所以构建会失败。但在此之前它会下载依赖项。？
• 由于我们预计构建会失败，因此我尝试了一种简单的技术来返回 0 并允许 docker 继续执行
• 这将加快后续构建流程，因为所有依赖项都已下载并且 docker 缓存了这一层。相比之下，Volume 挂载 gradle 缓存目录仍然是最好的方法。
• 上面的例子还展示了多阶段 docker 镜像构建，避免了多个 docker 构建文件。