在 Javascript 中使用正则表达式标记字符串
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/8441915/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Tokenizing strings using regular expression in Javascript
提问by Nawaz
Suppose I've a long string containing newlines and tabs as:
假设我有一个包含换行符和制表符的长字符串：
var x = "This is a long string.\n\t This is another one on next line.";
So how can we split this string into tokens, using regular expression?
那么我们如何使用正则表达式将该字符串拆分为标记呢？
I don't want to use .split(' ')because I want to learn Javascript's Regex.
我不想使用，.split(' ')因为我想学习 Javascript 的 Regex。
A more complicated string could be this:
一个更复杂的字符串可能是这样的：
var y = "This @is a #long $string. Alright, lets split this.";
Now I want to extract only the valid wordsout of this string, without special characters, and punctuation, i.e I want these:
现在我只想从这个字符串中提取有效的单词，没有特殊字符和标点符号，即我想要这些：
var xwords = ["This", "is", "a", "long", "string", "This", "is", "another", "one", "on", "next", "line"];
var ywords = ["This", "is", "a", "long", "string", "Alright", "lets", "split", "this"];
回答by Alexander Yezutov
Here is a jsfiddle example of what you asked: http://jsfiddle.net/ayezutov/BjXw5/1/
这是您所问的 jsfiddle 示例：http: //jsfiddle.net/ayezutov/BjXw5/1/
Basically, the code is very simple:
基本上，代码非常简单：
var y = "This @is a #long $string. Alright, lets split this.";
var regex = /[^\s]+/g; // This is "multiple not space characters, which should be searched not once in string"
var match = y.match(regex);
for (var i = 0; i<match.length; i++)
{
document.write(match[i]);
document.write('<br>');
}
UPDATE: Basically you can expand the list of separator characters: http://jsfiddle.net/ayezutov/BjXw5/2/
更新：基本上你可以扩展分隔符列表：http: //jsfiddle.net/ayezutov/BjXw5/2/
var regex = /[^\s\.,!?]+/g;
UPDATE 2:Only letters all the time: http://jsfiddle.net/ayezutov/BjXw5/3/
更新 2：一直只有字母：http: //jsfiddle.net/ayezutov/BjXw5/3/
var regex = /\w+/g;
回答by Prince John Wesley
Use \s+to tokenize the string.
使用\s+来标记字符串。
回答by kennebec
exec can loop through the matches to remove non-word (\W) characters.
exec 可以遍历匹配项以删除非单词 (\W) 字符。
var A= [], str= "This @is a #long $string. Alright, let's split this.",
rx=/\W*([a-zA-Z][a-zA-Z']*)(\W+|$)/g, words;
while((words= rx.exec(str))!= null){
A.push(words[1]);
}
A.join(', ')
/* returned value: (String)
This, is, a, long, string, Alright, let's, split, this
*/
回答by Kai
var words = y.split(/[^A-Za-z0-9]+/);
回答by Mar Cnu
Here is a solution using regex groups to tokenise the text using different types of tokens.
这是使用正则表达式组使用不同类型的标记对文本进行标记的解决方案。
You can test the code here https://jsfiddle.net/u3mvca6q/5/
你可以在这里测试代码https://jsfiddle.net/u3mvca6q/5/
/*
Basic Regex explanation:
/ Regex start
(\w+) First group, words \w means ASCII letter with \w + means 1 or more letters
| or
(,|!) Second group, punctuation
| or
(\s) Third group, white spaces
/ Regex end
g "global", enables looping over the string to capture one element at a time
Regex result:
result[0] : default group : any match
result[1] : group1 : words
result[2] : group2 : punctuation , !
result[3] : group3 : whitespace
*/
var basicRegex = /(\w+)|(,|!)|(\s)/g;
/*
Advanced Regex explanation:
[a-zA-Z\u0080-\u00FF] instead of \w Supports some Unicode letters instead of ASCII letters only. Find Unicode ranges here https://apps.timwhitlock.info/js/regex
(\.\.\.|\.|,|!|\?) Identify ellipsis (...) and points as separate entities
You can improve it by adding ranges for special punctuation and so on
*/
var advancedRegex = /([a-zA-Z\u0080-\u00FF]+)|(\.\.\.|\.|,|!|\?)|(\s)/g;
var basicString = "Hello, this is a random message!";
var advancedString = "Et en fran?ais ? Avec des caractères spéciaux ... With one point at the end.";
console.log("------------------");
var result = null;
do {
result = basicRegex.exec(basicString)
console.log(result);
} while(result != null)
console.log("------------------");
var result = null;
do {
result = advancedRegex.exec(advancedString)
console.log(result);
} while(result != null)
/*
Output:
Array [ "Hello", "Hello", undefined, undefined ]
Array [ ",", undefined, ",", undefined ]
Array [ " ", undefined, undefined, " " ]
Array [ "this", "this", undefined, undefined ]
Array [ " ", undefined, undefined, " " ]
Array [ "is", "is", undefined, undefined ]
Array [ " ", undefined, undefined, " " ]
Array [ "a", "a", undefined, undefined ]
Array [ " ", undefined, undefined, " " ]
Array [ "random", "random", undefined, undefined ]
Array [ " ", undefined, undefined, " " ]
Array [ "message", "message", undefined, undefined ]
Array [ "!", undefined, "!", undefined ]
null
*/
回答by awdz9nld
In order to extract word-only characters, we use the \wsymbol. Whether or not this will match Unicode characters or not is implementation-dependent, and you can use this referenceto see what the case is for your language/library.
为了提取纯单词字符，我们使用\w符号。这是否与 Unicode 字符匹配取决于实现，您可以使用此参考来查看您的语言/库的情况。
Please see Alexander Yezutov's answer (update 2) on how to apply this into an expression.
请参阅 Alexander Yezutov 关于如何将其应用于表达式的回答（更新 2）。
