bash shell 脚本对编码和行尾敏感吗？

Are shell scripts sensitive to encoding and line endings?

提问by thomasb

I am making a NW.js app on Mac, and want to run the app in dev mode by double-clicking on an icon. First step, I'm trying to make my shell script work.

我正在 Mac 上制作 NW.js 应用程序，并希望通过双击图标在开发模式下运行该应用程序。第一步，我试图让我的 shell 脚本工作。

Using VSCode on Windows (I wanted to gain time), I have created a run-nwfile at the root of my project, containing this:

在 Windows 上使用 VSCode（我想争取时间），我run-nw在项目的根目录下创建了一个文件，其中包含：

#!/bin/bash

cd "src"
npm install

cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &

but I get this output:

但我得到这个输出：

$ sh ./run-nw

: command not found  
: No such file or directory  
: command not found  
: No such file or directory  

Usage: npm <command>

where <command> is one of:  (snip commands list)

(snip npm help)

[email protected] /usr/local/lib/node_modules/npm  
: command not found  
: No such file or directory  
: command not found

I really don't understand:

我真的不明白：

• it seems that it takes empty lines as commands. In my editor (VSCode) I have tried to replace \r\nwith \n(in case the \rcreates problems) but it changes nothing.
• it seems that it doesn't find the folders (with or without the dirnameinstruction), or maybe it doesn't know about the cdcommand ?
• it seems that it doesn't understand the installargument to npm
• the part that really weirds me out, is that it still runs the app (if I did a npm installmanually)...

• 似乎它将空行作为命令。在我的编辑器（VSCode）中，我尝试替换\r\n为\n（以防\r产生问题），但它没有任何改变。
• 它似乎没有找到文件夹（有或没有dirname指令），或者它可能不知道cd命令？
• 似乎它不明白的install论点npm
• 真正让我感到奇怪的部分是它仍然运行该应用程序（如果我npm install手动执行）...

Not able to make it work properly, and suspecting something weird with the file itself, I created a new one directly on the Mac, using vim this time. I entered the exact same instructions, and... now it works without any issue.
A diff on the two files reveals exactly zero difference.

无法使其正常工作，并且怀疑文件本身有一些奇怪的东西，我直接在 Mac 上创建了一个新的，这次使用 vim。我输入了完全相同的说明，然后......现在它可以正常工作了。
两个文件的差异显示完全为零差异。

What can be the difference? What can make the first script not work? How can I find out?

有什么区别？什么会使第一个脚本不起作用？我怎样才能知道？

Update

更新

Following the accepted answer's recommandations, after the wrong line endings came back, I checked multiple things. It turns out that since I copied my ~/.gitconfigfrom my Windows machine, I had autocrlf=true, so every time I modified the bash file under Windows, it re-set the line endings to \r\n.
So, in addition to running dos2unix (which you will have to install using Homebrew on mac), if you're using Git, check your config.

按照接受的答案的建议，在错误的行结尾回来后，我检查了很多东西。原来，自从我复制我~/.gitconfig从我的Windows机器，我有autocrlf=true，所以每次我修改了庆典文件在Windows下时，它重新设置行结束\r\n。
因此，除了运行 dos2unix（您必须在 Mac 上使用 Homebrew 安装）之外，如果您使用的是 Git，请检查您的配置。

回答by Anthony Geoghegan

Yes. Bash scripts aresensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).

是的。Bash 脚本对行尾很敏感，无论是在脚本本身还是在它处理的数据中。它们应该有 Unix 风格的行尾，即每行都以换行符（十进制 10，十六进制 0A ASCII）结束。

DOS/Windows line endings in the script

脚本中的 DOS/Windows 行结尾

With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as

对于 Windows 或 DOS 样式的行结尾，每行都以回车符结束，后跟换行符。如果脚本文件以 Windows 行结尾保存，则 Bash 将该文件视为

#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M

Note: I've used caret notation to represent non-printing characters, i.e., ^Mis used to to represent the Carriage Return characters (represented as \rin other contexts); this is the same technique used by cat -vand Vim.

注意：我已经使用插入符号来表示非打印字符，即^M用于表示回车符（\r在其他上下文中表示）；这cat -v与 Vim使用的技术相同。

In this case, the carriage return (^Mor \r) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.

在这种情况下，回车（^M或\r）不被视为空格。Bash 将 shebang 之后的第一行（由单个回车符组成）解释为要运行的命令/程序的名称。

• Since there is no command named ^M, it prints : command not found
• Since there is no directory named "src"^M(or src^M), it prints : No such file or directory
• It passes install^Minstead of installas an argument to npmwhich causes npmto complain.

• 由于没有名为 的命令^M，它打印: command not found
• 由于没有名为"src"^M（或src^M）的目录，它打印: No such file or directory
• 它传递install^M而不是install作为npm导致npm抱怨的参数。

DOS/Windows line endings in input data

输入数据中的 DOS/Windows 行结尾

Like above, if you have an input file with carriage returns:

像上面一样，如果你有一个带回车的输入文件：

hello^M
world^M

then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grepwill fail to find lines that are obviously there:

那么它在编辑器中和将其写入屏幕时看起来会完全正常，但工具可能会产生奇怪的结果。例如，grep将无法找到明显存在的行：

$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)

Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:

附加的文本将改写该行，因为回车将光标移动到该行的开头：

$ sed -e 's/$/!/' file.txt
!ello
!orld

String comparison will seem to fail, even though strings appear to be the same when writing to screen:

字符串比较似乎会失败，即使写入屏幕时字符串看起来相同：

$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
  then echo "Variables are equal."
  else echo "Sorry, $a is not equal to $b"
  fi

Sorry, hello is not equal to hello

Solutions

解决方案

The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:

解决方案是将文件转换为使用 Unix 样式的行结尾。有多种方法可以实现这一点：

1. This can be done using the dos2unixprogram:

   dos2unix filename
   

2. Open the file in a capabletext editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:

   :set fileformat=unix
   

3. If you have a version of the sedutility that supports the -ior --in-placeoption, e.g., GNU sed, you could run the following command to strip trailing carriage returns:

   sed -i 's/\r$//' filename
   

   With other versions of sed, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).

   sed 's/\r$//' filename > filename.unix
   

4. Similarly, the trtranslation filter can be used to delete unwanted characters from its input:

   tr -d '\r' <filename >filename.unix
   

1. 这可以使用以下dos2unix程序完成：

   dos2unix filename
   

2. 在功能强大的文本编辑器（Sublime、Notepad++，而不是 Notepad）中打开文件并将其配置为以 Unix 行结尾保存文件，例如，使用 Vim，在（重新）保存之前运行以下命令：

   :set fileformat=unix
   

3. 如果您有一个sed支持-ior--in-place选项的实用程序版本，例如 GNU sed，您可以运行以下命令来去除尾随回车：

   sed -i 's/\r$//' filename
   

   对于其他版本的sed，您可以使用输出重定向来写入新文件。确保为重定向目标使用不同的文件名（以后可以重命名）。

   sed 's/\r$//' filename > filename.unix
   

4. 同样，tr翻译过滤器可用于从其输入中删除不需要的字符：

   tr -d '\r' <filename >filename.unix
   

Cygwin Bash

赛格温·巴什

With the Bash port for Cygwin, there's a custom igncroption that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). This can be enabled for the currentshell by running set -o igncr.

对于 Cygwin 的 Bash 端口，有一个自定义igncr选项可以设置为忽略行尾中的回车（大概是因为它的许多用户使用本机 Windows 程序来编辑他们的文本文件）。这可以通过运行为当前shell启用set -o igncr。

Setting this option applies only to the currentshell process so it can be useful when sourcingfiles with extraneous carriage returns. If you regularly encounter shell scripts with DOS line endings and want this option to be set permanently, you could set an environment variable called SHELLOPTS(all capital letters) to include igncr. This environment variable is used by Bash to set shell options when it starts (before reading any startup files).

设置此选项仅适用于当前的shell 进程，因此在获取带有无关回车符的文件时很有用。如果您经常遇到带有 DOS 行结尾的 shell 脚本并希望永久设置此选项，则可以设置一个名为SHELLOPTS（全部大写字母）的环境变量以包含igncr. Bash 在启动时（在读取任何启动文件之前）使用这个环境变量来设置 shell 选项。

Useful utilities

有用的工具

The fileutility is useful for quickly seeing which line endings are used in a text file. Here's what it prints for for each file type:

该file实用程序可用于快速查看文本文件中使用了哪些行结尾。这是它为每种文件类型打印的内容：

• Unix line endings: Bourne-Again shell script, ASCII text executable
• Mac line endings: Bourne-Again shell script, ASCII text executable, with CR line terminators
• DOS line endings: Bourne-Again shell script, ASCII text executable, with CRLF line terminators

• Unix 行结尾： Bourne-Again shell script, ASCII text executable
• Mac 行结尾： Bourne-Again shell script, ASCII text executable, with CR line terminators
• DOS 行结尾： Bourne-Again shell script, ASCII text executable, with CRLF line terminators

The GNU version of the catutility has a -v, --show-nonprintingoption that displays non-printing characters.

该cat实用程序的 GNU 版本具有-v, --show-nonprinting显示非打印字符的选项。

The dos2unixutility is specifically written for converting text files between Unix, Mac and DOS line endings.

该dos2unix实用程序专门用于在 Unix、Mac 和 DOS 行尾之间转换文本文件。

Useful links

有用的链接

Wikipedia has an excellent articlecovering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).

维基百科有一篇出色的文章，涵盖了标记文本行结尾的多种不同方式、此类编码的历史以及不同操作系统、编程语言和 Internet 协议（例如 FTP）中如何处理换行符。

Files with classic Mac OS line endings

带有经典 Mac OS 行尾的文件

With Classic Mac OS(pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:

在经典 Mac OS（OS X 之前）中，每一行都以回车符（十进制 13，十六进制 0D 的 ASCII）结束。如果脚本文件以这样的行结尾保存，Bash 只会看到像这样的一长行：

#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M

Since this single long line begins with an octothorpe (#), Bash treats the line (and the whole file) as a single comment.

由于这一长行以 octothorpe ( #)开头，因此 Bash将这一行（以及整个文件）视为单个注释。

Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEPoperating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it's worthwhile to show how Bash would attempt to interpret such files.

注：2001 年，Apple 推出了基于 BSD 衍生的NeXTSTEP操作系统的Mac OS X。因此，OS X 也使用 Unix 风格的 LF-only 行尾，从那时起，以 CR 结尾的文本文件变得极为罕见。尽管如此，我认为展示 Bash 将如何尝试解释这些文件是值得的。

回答by CONvid19

On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to clickon CRLF/LFto togglebetween the two types of line separators (\r\nand \n).

在 JetBrains 产品（PyCharm、PHPStorm、IDEA 等）上，您需要click打开CRLF/LF以在两种类型的行分隔符（和）之间切换。\r\n\n

回答by Igor Soudakevitch

One more way to get rid of the unwanted CR ('\r') character is to run the trcommand, for example:

摆脱不需要的 CR ('\r') 字符的另一种方法是运行tr命令，例如：

$ tr -d '\r' < dosScript.py > nixScript.py

回答by tripleee

Coming from a duplicate, if the problem is that you have files whose namescontain ^Mat the end, you can rename them with

来自重复，如果问题是您的文件名称包含^M在末尾，您可以使用

for f in *$'\r'; do
    mv "$f" "${f%$'\r'}"
done

You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unixed and then rerun?) but sometimes this is not feasible.

您首先要修复导致这些文件名称损坏的任何原因（可能是创建它们的脚本应该被dos2unix编辑然后重新运行？）但有时这是不可行的。

The $'\r'syntax is Bash-specific; if you have a different shell, maybe you need to use some other notation. Perhaps see also Difference between sh and bash

该$'\r'语法是击特定的; 如果您有不同的外壳，也许您需要使用其他一些表示法。也许另见sh 和 bash 的区别

回答by danR

The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.

MAC / Linux 上最简单的方法 - 使用“touch”命令创建文件，使用 VI 或 VIM 编辑器打开此文件，粘贴代码并保存。这将自动删除 Windows 字符。