node.js 避免在弹性 beantalk 中重建 node_modules

Question

提问by Kirill Kay

We have a fairly simple node.js app, but due to AWS Elastic Beanstalk deployment mechanism, it takes about 5 minutes to roll-out a new version (via git aws.push) even after a single file commit.

我们有一个相当简单的 node.js 应用程序，但由于 AWS Elastic Beanstalk 部署机制，git aws.push即使在提交单个文件后，也需要大约 5 分钟才能推出新版本（通过）。

I.e. the commit itself (and upload) is fast (only 1 file to push), but then Elastic Beanstalk fetches whole package from S3, unzips it and runs npm install, which causes node-gyp to compile some modules. Upon installation/building completion, Elastic Beanstalk wipes /var/app/currentand replaces it with the new app version.

即提交本身（和上传）很快（只有 1 个文件要推送），但随后 Elastic Beanstalk 从 S3 获取整个包，解压缩并运行npm install，这会导致 node-gyp 编译一些模块。安装/构建完成后，Elastic Beanstalk 将擦除/var/app/current并将其替换为新的应用程序版本。

Needless to say, constant node_modules rebuilding is not necessary, and rebuilding that takes 30 seconds on my old Macbook Air, takes >5 mins on a ec2.micro instance, not fun.

不用说，不断的 node_modules 重建是没有必要的，在我的旧 Macbook Air 上重建需要 30 秒，在 ec2.micro 实例上需要 5 分钟以上，这并不有趣。

I see two approaches here:

我在这里看到两种方法：

tweak /opt/containerfiles/ebnode.pyand play with node_modules location to avoid its removal and rebuilding upon deployment.
set up a git repo on Elastic Beanstalk EC2 instance and basically re-write deployment procedure ourselves, so /var/app/current receives pushes and runs npm installonly when necessary (which makes Elastic Beanstalk to look like OpsWorks..)

调整/opt/containerfiles/ebnode.py和使用 node_modules 位置，以避免在部署时将其删除和重建。
在 Elastic Beanstalk EC2 实例上设置一个 git repo 并基本上自己重写部署过程，因此 /var/app/current 接收推送并npm install仅在必要时运行（这使得 Elastic Beanstalk 看起来像 OpsWorks ..）

Both options lack grace and are prone to issues when Amazon updates their Elastic Beanstalk hooks and architecture.

当 Amazon 更新其 Elastic Beanstalk 挂钩和架构时，这两种选择都缺乏优雅，并且容易出现问题。

Maybe somebody has a better idea how to avoid constant rebuilding of node_modules that are already present in the app dir? Thank you.

也许有人有更好的主意如何避免不断重建应用程序目录中已经存在的 node_modules？谢谢你。

Answer 1

采纳答案by Tronix117

Thanks Kirill, it was really helpful !

谢谢基里尔，这真的很有帮助！

I'm just sharing my config file for people who just look the simple solution to the npm install. This file needs to be placed in the .ebextensionsfolder of the project, it is lighter since it doesn't include last version of node installation, and ready to use.

我只是将我的配置文件分享给那些只是寻找npm install. 这个文件需要放在.ebextensions项目的文件夹中，它比较轻，因为它不包括最新版本的节点安装，并且可以使用。

It also dynamically checks the node version installed, so no need for it to be included in the env.vars file.

它还动态检查安装的节点版本，因此无需将其包含在 env.vars 文件中。

.ebextensions/00_deploy_npm.config

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    content: |
      export NPM_CONFIG_LOGLEVEL=error
      export NODE_PATH=`ls -td /opt/elasticbeanstalk/node-install/node-* | head -1`/bin
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    content: |
      #!/bin/bash
      . /opt/elasticbeanstalk/env.vars
      function error_exit
      {
        eventHelper.py --msg "" --severity ERROR
        exit 
      }

      #install not-installed yet app node_modules
      if [ ! -d "/var/node_modules" ]; then
        mkdir /var/node_modules ;
      fi
      if [ -d /tmp/deployment/application ]; then
        ln -s /var/node_modules /tmp/deployment/application/
      fi

      OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && $NODE_PATH/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy

Answer 2

回答by Kirill Kay

25/01/13 NOTE: updated scripts to run npm -g version upgrade (only once, on initial instance roll out or rebuild) and to avoid NPM operations during EB configuration change (when app dir is not present, to avoid error and to speed up configuration updates).

25/01/13 注意：更新脚本以运行 npm -g 版本升级（仅一次，在初始实例推出或重建时）并避免 EB 配置更改期间的 NPM 操作（当应用程序目录不存在时，以避免错误和加快配置更新）。

Okay, Elastic Beanstalk behaves dodgy with recent node.js builds (including presumably supported v.0.10.10), so I decided to go ahead and tweak EB to do the following:

好的，Elastic Beanstalk 对最近的 node.js 构建（包括可能支持的 v.0.10.10）表现得很不可靠，所以我决定继续调整 EB 以执行以下操作：

to install ANY node.js version as per your env.config (including the most recent ones that are not yet supported by AWS EB)
to avoid rebuilding existing node modules, including in-app node_modules dir
to install node.js globally (and any desired module as well).

根据您的 env.config 安装任何 node.js 版本（包括 AWS EB 尚不支持的最新版本）
避免重建现有的节点模块，包括应用内 node_modules 目录
全局安装 node.js（以及任何所需的模块）。

Basically, I use env.config to replace deploy&config hooks with customized ones (see below). Also, in a default EB container setup some env variables are missing ($HOMEfor example) and node-gypsometimes fails during rebuild because of it (took me 2 hours of googling and reinstalling libxmljs to resolve this).

基本上，我使用 env.config 用自定义钩子替换 deploy&config 钩子（见下文）。此外，在默认的 EB 容器设置中，一些 env 变量丢失（$HOME例如）并且node-gyp有时会因此在重建过程中失败（我花了 2 个小时的谷歌搜索并重新安装 libxmljs 来解决这个问题）。

Below are the files to be included along with your build. You can inject them via env.config as inline code or via source: URL(as in this example)

以下是与您的构建一起包含的文件。您可以通过 env.config 作为内联代码或通过source: URL（如本例所示）注入它们

env.vars(desired node version & arch are included here and in env.config, see below)

env.vars（所需的节点版本和架构包含在此处和 env.config 中，见下文）

export HOME=/root
export NPM_CONFIG_LOGLEVEL=error
export NODE_VER=0.10.24
export ARCH=x86
export PATH="$PATH:/opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/:/root/.npm"

40install_node.sh(fetch and ungzip desired node.js version, make global symlinks, update global npm version)

40install_node.sh（获取并解压缩所需的 node.js 版本，制作全局符号链接，更新全局 npm 版本）

#!/bin/bash
#source env variables including node version
. /opt/elasticbeanstalk/env.vars

function error_exit
{
  eventHelper.py --msg "" --severity ERROR
  exit 
}

#UNCOMMENT to update npm, otherwise will be updated on instance init or rebuild
#rm -f /opt/elasticbeanstalk/node-install/npm_updated

#download and extract desired node.js version
OUT=$( [ ! -d "/opt/elasticbeanstalk/node-install" ] && mkdir /opt/elasticbeanstalk/node-install ; cd /opt/elasticbeanstalk/node-install/ && wget -nc http://nodejs.org/dist/v$NODE_VER/node-v$NODE_VER-linux-$ARCH.tar.gz && tar --skip-old-files -xzpf node-v$NODE_VER-linux-$ARCH.tar.gz) || error_exit "Failed to UPDATE node version. $OUT" $?.
echo $OUT

#make sure node binaries can be found globally
if [ ! -L /usr/bin/node ]; then
  ln -s /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/node /usr/bin/node
fi

if [ ! -L /usr/bin/npm ]; then
ln -s /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm /usr/bin/npm
fi

if [ ! -f "/opt/elasticbeanstalk/node-install/npm_updated" ]; then
/opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/ && /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm update npm -g
touch /opt/elasticbeanstalk/node-install/npm_updated
echo "YAY! Updated global NPM version to `npm -v`"
else
  echo "Skipping NPM -g version update. To update, please uncomment 40install_node.sh:12"
fi

50npm.sh(creates /var/node_modules, symlinks it to app dir and runs npm install. You can install any module globally from here, they will land in /root/.npm)

50npm.sh（创建 /var/node_modules，将其符号链接到应用程序目录并运行 npm install。您可以从这里全局安装任何模块，它们将位于 /root/.npm 中）

#!/bin/bash
. /opt/elasticbeanstalk/env.vars
function error_exit
{
  eventHelper.py --msg "" --severity ERROR
  exit 
}

#install not-installed yet app node_modules
if [ ! -d "/var/node_modules" ]; then
  mkdir /var/node_modules ;
fi
if [ -d /tmp/deployment/application ]; then
  ln -s /var/node_modules /tmp/deployment/application/
fi

OUT=$([ -d "/tmp/deployment/application" ] && cd /tmp/deployment/application && /opt/elasticbeanstalk/node-install/node-v$NODE_VER-linux-$ARCH/bin/npm install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
echo $OUT

env.config(note node version here too, and to be safe, put desired node version in env config in AWS console as well. I'm not certain which of these settings will take precedence.)

env.config（注意这里的节点版本，为了安全起见，将所需的节点版本也放在 AWS 控制台的 env config 中。我不确定这些设置中的哪一个会优先。）

packages:
  yum:
    git: []
    gcc: []
    make: []
    openssl-devel: []

option_settings:
  - option_name: NODE_ENV
    value: production
  - option_name: RDS_HOSTNAME
    value: fill_me_in
  - option_name: RDS_PASSWORD
    value: fill_me_in
  - option_name: RDS_USERNAME
    value: fill_me_in
  - namespace: aws:elasticbeanstalk:container:nodejs
    option_name: NodeVersion
    value: 0.10.24

files:
  "/opt/elasticbeanstalk/env.vars" :
    mode: "000775"
    owner: root
    group: users
    source: https://dl.dropbox.com/....
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/40install_node.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....
  "/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh" :
    mode: "000666"
    owner: root
    group: users
    content: |
       #no need to run npm install during configdeploy
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/40install_node.sh" :
    mode: "000775"
    owner: root
    group: users
    source: https://raw.github.com/....

There you have it: on t1.micro instance deployment now takes 20-30 secs instead of 10-15 minutes! If you deploy 10 times a day, this tweak will save you 3 (three) weeks in a year. Hope it helps and special thanks to AWS EB staff for my lost weekend :)

你有它：在 t1.micro 实例部署现在需要 20-30 秒而不是 10-15 分钟！如果您每天部署 10 次，此调整将在一年中为您节省 3（三）周。希望它对我失去的周末有所帮助并特别感谢 AWS EB 工作人员 :)

Answer 3

回答by panK

There's npm package that's overwriting default EB behaviour for npm installcommand by truncating following files:

有一个 npm 包npm install通过截断以下文件来覆盖命令的默认 EB 行为：

/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh
/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh

/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh
/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh

https://www.npmjs.com/package/eb-disable-npm

Might be better than just copying script from SO, since this package is maintained and probably will be updated when EB behaviour will change.

可能比仅仅从 SO 复制脚本更好，因为这个包得到维护并且可能会在 EB 行为发生变化时更新。

Answer 4

回答by Peter

I've found a quick solution to this. I looked through the build scripts that Amazon are using and they only run npm installif package.json is present. So after your initial deploy you can change it to _package.jsonand npm installwon't run anymore! It's not the best solution but it's a quick fix if you need one!

我找到了一个快速解决方案。我查看了 Amazon 使用的构建脚本，它们仅npm install在 package.json 存在时运行。因此，在初始部署后，您可以将其更改为_package.json并且npm install不再运行！这不是最好的解决方案，但如果您需要，这是一个快速解决方案！

Answer 5

回答by Eric

I had 10+ minute builds when I would deploy. The solution was much simpler than others have came up with... Just check node_modules into git! See http://www.futurealoof.com/posts/nodemodules-in-git.htmlfor the reasoning

我在部署时进行了 10 分钟以上的构建。解决方案比其他人提出的要简单得多......只需将 node_modules 签入 git 即可！有关推理，请参阅http://www.futurealoof.com/posts/nodemodules-in-git.html

node.js 避免在弹性 beantalk 中重建 node_modules

提问by Kirill Kay

采纳答案by Tronix117

回答by Kirill Kay

回答by panK

回答by Peter

回答by Eric

相关推荐

最近更新

标签

node.js 避免在弹性 beantalk 中重建 node_modules

提问by Kirill Kay

采纳答案by Tronix117

回答by Kirill Kay

回答by panK

回答by Peter

回答by Eric

相关推荐

webSocketServer node.js 如何区分客户端

index.js 在 node.js 项目中的用途是什么？

如何将 node.js 模块作为 node.js 程序的子进程执行？

node.js 如何在 express.js 中限制上传文件大小

相关推荐

最近更新

标签