node.js 在 Mongoose 中填充后查询

Querying after populate in Mongoose

提问by jschr

I'm pretty new to Mongoose and MongoDB in general so I'm having a difficult time figuring out if something like this is possible:

总的来说，我对 Mongoose 和 MongoDB 还很陌生，所以我很难弄清楚这样的事情是否可行：

Item = new Schema({
    id: Schema.ObjectId,
    dateCreated: { type: Date, default: Date.now },
    title: { type: String, default: 'No Title' },
    description: { type: String, default: 'No Description' },
    tags: [ { type: Schema.ObjectId, ref: 'ItemTag' }]
});

ItemTag = new Schema({
    id: Schema.ObjectId,
    tagId: { type: Schema.ObjectId, ref: 'Tag' },
    tagName: { type: String }
});



var query = Models.Item.find({});

query
    .desc('dateCreated')
    .populate('tags')
    .where('tags.tagName').in(['funny', 'politics'])
    .run(function(err, docs){
       // docs is always empty
    });

Is there a better way do this?

有没有更好的方法来做到这一点？

Edit

编辑

Apologies for any confusion. What I'm trying to do is get all Items that contain either the funny tag or politics tag.

对任何混淆表示歉意。我想要做的是获取所有包含有趣标签或标签的项目。

Edit

编辑

Document without where clause:

没有 where 子句的文档：

[{ 
    _id: 4fe90264e5caa33f04000012,
    dislikes: 0,
    likes: 0,
    source: '/uploads/loldog.jpg',
    comments: [],
    tags: [{
        itemId: 4fe90264e5caa33f04000012,
        tagName: 'movies',
        tagId: 4fe64219007e20e644000007,
        _id: 4fe90270e5caa33f04000015,
        dateCreated: Tue, 26 Jun 2012 00:29:36 GMT,
        rating: 0,
        dislikes: 0,
        likes: 0 
    },
    { 
        itemId: 4fe90264e5caa33f04000012,
        tagName: 'funny',
        tagId: 4fe64219007e20e644000002,
        _id: 4fe90270e5caa33f04000017,
        dateCreated: Tue, 26 Jun 2012 00:29:36 GMT,
        rating: 0,
        dislikes: 0,
        likes: 0 
    }],
    viewCount: 0,
    rating: 0,
    type: 'image',
    description: null,
    title: 'dogggg',
    dateCreated: Tue, 26 Jun 2012 00:29:24 GMT 
 }, ... ]

With the where clause, I get an empty array.

使用 where 子句，我得到一个空数组。

采纳答案by Neil Lunn

With a modern MongoDB greater than 3.2 you can use $lookupas an alternate to .populate()in most cases. This also has the advantage of actually doing the join "on the server" as opposed to what .populate()does which is actually "multiple queries" to "emulate"a join.

使用大于 3.2 的现代 MongoDB，您可以在大多数情况下$lookup用作替代.populate()。这也具有实际在“服务器上”进行连接的优点，而不是.populate()实际上是“多个查询”来“模拟”连接。

So .populate()is notreally a "join" in the sense of how a relational database does it. The $lookupoperator on the other hand, actually does the work on the server, and is more or less analogous to a "LEFT JOIN":

因此，.populate()就关系数据库的工作方式而言，这并不是真正的“连接”。在$lookup另一方面，运营商，实际执行服务器上的工作，是一个或多或少类似“LEFT JOIN”：

Item.aggregate(
  [
    { "$lookup": {
      "from": ItemTags.collection.name,
      "localField": "tags",
      "foreignField": "_id",
      "as": "tags"
    }},
    { "$unwind": "$tags" },
    { "$match": { "tags.tagName": { "$in": [ "funny", "politics" ] } } },
    { "$group": {
      "_id": "$_id",
      "dateCreated": { "$first": "$dateCreated" },
      "title": { "$first": "$title" },
      "description": { "$first": "$description" },
      "tags": { "$push": "$tags" }
    }}
  ],
  function(err, result) {
    // "tags" is now filtered by condition and "joined"
  }
)

N.B.The .collection.namehere actually evaluates to the "string" that is the actual name of the MongoDB collection as assigned to the model. Since mongoose "pluralizes" collection names by default and $lookupneeds the actual MongoDB collection name as an argument ( since it's a server operation ), then this is a handy trick to use in mongoose code, as opposed to "hard coding" the collection name directly.

NB在.collection.name这里实际上是计算结果为“字符串”这是MongoDB的集合的实际名称为分配给模型。由于 mongoose 默认情况下“复数化”集合名称并且$lookup需要实际的 MongoDB 集合名称作为参数（因为它是服务器操作），因此这是在 mongoose 代码中使用的一个方便的技巧，而不是直接“硬编码”集合名称.

Whilst we could also use $filteron arrays to remove the unwanted items, this is actually the most efficient form due to Aggregation Pipeline Optimizationfor the special condition of as $lookupfollowed by both an $unwindand a $matchcondition.

虽然我们也可以使用$filter数组来删除不需要的项目，但这实际上是最有效的形式，因为聚合管道优化用于特殊条件 as$lookup后跟 an$unwind和 a$match条件。

This actually results in the three pipeline stages being rolled into one:

这实际上导致三个管道阶段被合并为一个：

   { "$lookup" : {
     "from" : "itemtags",
     "as" : "tags",
     "localField" : "tags",
     "foreignField" : "_id",
     "unwinding" : {
       "preserveNullAndEmptyArrays" : false
     },
     "matching" : {
       "tagName" : {
         "$in" : [
           "funny",
           "politics"
         ]
       }
     }
   }}

This is highly optimal as the actual operation "filters the collection to join first", then it returns the results and "unwinds" the array. Both methods are employed so the results do not break the BSON limit of 16MB, which is a constraint that the client does not have.

这是高度优化的，因为实际操作“首先过滤要加入的集合”，然后返回结果并“展开”数组。两种方法都被采用，因此结果不会打破 16MB 的 BSON 限制，这是客户端没有的约束。

The only problem is that it seems "counter-intuitive" in some ways, particularly when you want the results in an array, but that is what the $groupis for here, as it reconstructs to the original document form.

唯一的问题是它在某些方面似乎“违反直觉”，特别是当您想要数组中的结果时，但这$group就是这里的目的，因为它重建为原始文档形式。

It's also unfortunate that we simply cannot at this time actually write $lookupin the same eventual syntax the server uses. IMHO, this is an oversight to be corrected. But for now, simply using the sequence will work and is the most viable option with the best performance and scalability.

同样不幸的是，我们此时根本无法实际编写$lookup与服务器使用的相同的最终语法。恕我直言，这是一个需要纠正的疏忽。但就目前而言，简单地使用该序列是可行的，并且是具有最佳性能和可扩展性的最可行的选择。

Addendum - MongoDB 3.6 and upwards

附录 - MongoDB 3.6 及更高版本

Though the pattern shown here is fairly optimizeddue to how the other stages get rolled into the $lookup, it does have one failing in that the "LEFT JOIN" which is normally inherent to both $lookupand the actions of populate()is negated by the "optimal"usage of $unwindhere which does not preserve empty arrays. You can add the preserveNullAndEmptyArraysoption, but this negates the "optimized"sequence described above and essentially leaves all three stages intact which would normally be combined in the optimization.

尽管这里显示的模式由于其他阶段如何被卷入 中而得到了相当优化$lookup，但它确实有一个失败，即“LEFT JOIN”通常是两者固有的，$lookup并且 的操作populate()被“最佳”使用所否定$unwind这里不保留空数组。您可以添加该preserveNullAndEmptyArrays选项，但这会否定上述“优化”序列，并且基本上保持所有三个阶段完整无缺，这通常会在优化中组合。

MongoDB 3.6 expands with a "more expressive"form of $lookupallowing a "sub-pipeline" expression. Which not only meets the goal of retaining the "LEFT JOIN" but still allows an optimal query to reduce results returned and with a much simplified syntax:

MongoDB 3.6 扩展为允许“子管道”表达式的“更具表现力”的形式$lookup。这不仅满足了保留“LEFT JOIN”的目标，而且仍然允许优化查询以减少返回的结果并使用更简化的语法：

Item.aggregate([
  { "$lookup": {
    "from": ItemTags.collection.name,
    "let": { "tags": "$tags" },
    "pipeline": [
      { "$match": {
        "tags": { "$in": [ "politics", "funny" ] },
        "$expr": { "$in": [ "$_id", "$$tags" ] }
      }}
    ]
  }}
])

The $exprused in order to match the declared "local" value with the "foreign" value is actually what MongoDB does "internally" now with the original $lookupsyntax. By expressing in this form we can tailor the initial $matchexpression within the "sub-pipeline" ourselves.

在$expr以匹配使用的已声明与“洋”价值“本地”值实际上是MongoDB中做什么“内部”现在与原来的$lookup语法。通过以这种形式表达，我们可以自己定制$match“子管道”中的初始表达。

In fact, as a true "aggregation pipeline" you can do just about anything you can do with an aggregation pipeline within this "sub-pipeline" expression, including "nesting" the levels of $lookupto other related collections.

事实上，作为一个真正的“聚合管道”，您可以在这个“子管道”表达式中对聚合管道做任何可以做的事情，包括“嵌套”$lookup其他相关集合的级别。

Further usage is a bit beyond the scope of what the question here asks, but in relation to even "nested population" then the new usage pattern of $lookupallows this to be much the same, and a "lot"more powerful in it's full usage.

进一步的使用有点超出了这里问题的范围，但对于即使是“嵌套人口”，新的使用模式也$lookup允许它大致相同，并且在它的完整使用中“很多”更强大。

---

---

Working Example

工作示例

The following gives an example using a static method on the model. Once that static method is implemented the call simply becomes:

下面给出了在模型上使用静态方法的示例。一旦实现了该静态方法，调用就变成了：

  Item.lookup(
    {
      path: 'tags',
      query: { 'tags.tagName' : { '$in': [ 'funny', 'politics' ] } }
    },
    callback
  )

Or enhancing to be a bit more modern even becomes:

或者增强一点现代感甚至变成：

  let results = await Item.lookup({
    path: 'tags',
    query: { 'tagName' : { '$in': [ 'funny', 'politics' ] } }
  })

Making it very similar to .populate()in structure, but it's actually doing the join on the server instead. For completeness, the usage here casts the returned data back to mongoose document instances at according to both the parent and child cases.

使其与.populate()结构非常相似，但它实际上是在服务器上进行连接。为完整起见，此处的用法根据父案例和子案例将返回的数据转换回 mongoose 文档实例。

It's fairly trivial and easy to adapt or just use as is for most common cases.

对于大多数常见情况，它相当简单且易于适应或仅使用。

N.BThe use of asynchere is just for brevity of running the enclosed example. The actual implementation is free of this dependency.

注意这里使用async只是为了运行所附示例的简洁。实际的实现没有这种依赖性。

const async = require('async'),
      mongoose = require('mongoose'),
      Schema = mongoose.Schema;

mongoose.Promise = global.Promise;
mongoose.set('debug', true);
mongoose.connect('mongodb://localhost/looktest');

const itemTagSchema = new Schema({
  tagName: String
});

const itemSchema = new Schema({
  dateCreated: { type: Date, default: Date.now },
  title: String,
  description: String,
  tags: [{ type: Schema.Types.ObjectId, ref: 'ItemTag' }]
});

itemSchema.statics.lookup = function(opt,callback) {
  let rel =
    mongoose.model(this.schema.path(opt.path).caster.options.ref);

  let group = { "$group": { } };
  this.schema.eachPath(p =>
    group.$group[p] = (p === "_id") ? "$_id" :
      (p === opt.path) ? { "$push": `$${p}` } : { "$first": `$${p}` });

  let pipeline = [
    { "$lookup": {
      "from": rel.collection.name,
      "as": opt.path,
      "localField": opt.path,
      "foreignField": "_id"
    }},
    { "$unwind": `$${opt.path}` },
    { "$match": opt.query },
    group
  ];

  this.aggregate(pipeline,(err,result) => {
    if (err) callback(err);
    result = result.map(m => {
      m[opt.path] = m[opt.path].map(r => rel(r));
      return this(m);
    });
    callback(err,result);
  });
}

const Item = mongoose.model('Item', itemSchema);
const ItemTag = mongoose.model('ItemTag', itemTagSchema);

function log(body) {
  console.log(JSON.stringify(body, undefined, 2))
}
async.series(
  [
    // Clean data
    (callback) => async.each(mongoose.models,(model,callback) =>
      model.remove({},callback),callback),

    // Create tags and items
    (callback) =>
      async.waterfall(
        [
          (callback) =>
            ItemTag.create([{ "tagName": "movies" }, { "tagName": "funny" }],
              callback),

          (tags, callback) =>
            Item.create({ "title": "Something","description": "An item",
              "tags": tags },callback)
        ],
        callback
      ),

    // Query with our static
    (callback) =>
      Item.lookup(
        {
          path: 'tags',
          query: { 'tags.tagName' : { '$in': [ 'funny', 'politics' ] } }
        },
        callback
      )
  ],
  (err,results) => {
    if (err) throw err;
    let result = results.pop();
    log(result);
    mongoose.disconnect();
  }
)

Or a little more modern for Node 8.x and above with async/awaitand no additional dependencies:

或者对于 Node 8.x 及更高版本更现代一些async/await，没有额外的依赖：

const { Schema } = mongoose = require('mongoose');
const uri = 'mongodb://localhost/looktest';

mongoose.Promise = global.Promise;
mongoose.set('debug', true);

const itemTagSchema = new Schema({
  tagName: String
});

const itemSchema = new Schema({
  dateCreated: { type: Date, default: Date.now },
  title: String,
  description: String,
  tags: [{ type: Schema.Types.ObjectId, ref: 'ItemTag' }]
});

itemSchema.statics.lookup = function(opt) {
  let rel =
    mongoose.model(this.schema.path(opt.path).caster.options.ref);

  let group = { "$group": { } };
  this.schema.eachPath(p =>
    group.$group[p] = (p === "_id") ? "$_id" :
      (p === opt.path) ? { "$push": `$${p}` } : { "$first": `$${p}` });

  let pipeline = [
    { "$lookup": {
      "from": rel.collection.name,
      "as": opt.path,
      "localField": opt.path,
      "foreignField": "_id"
    }},
    { "$unwind": `$${opt.path}` },
    { "$match": opt.query },
    group
  ];

  return this.aggregate(pipeline).exec().then(r => r.map(m => 
    this({ ...m, [opt.path]: m[opt.path].map(r => rel(r)) })
  ));
}

const Item = mongoose.model('Item', itemSchema);
const ItemTag = mongoose.model('ItemTag', itemTagSchema);

const log = body => console.log(JSON.stringify(body, undefined, 2));

(async function() {
  try {

    const conn = await mongoose.connect(uri);

    // Clean data
    await Promise.all(Object.entries(conn.models).map(([k,m]) => m.remove()));

    // Create tags and items
    const tags = await ItemTag.create(
      ["movies", "funny"].map(tagName =>({ tagName }))
    );
    const item = await Item.create({ 
      "title": "Something",
      "description": "An item",
      tags 
    });

    // Query with our static
    const result = (await Item.lookup({
      path: 'tags',
      query: { 'tags.tagName' : { '$in': [ 'funny', 'politics' ] } }
    })).pop();
    log(result);

    mongoose.disconnect();

  } catch (e) {
    console.error(e);
  } finally {
    process.exit()
  }
})()

And from MongoDB 3.6 and upward, even without the $unwindand $groupbuilding:

从 MongoDB 3.6 及更高版本开始，即使没有$unwind和$group构建：

const { Schema, Types: { ObjectId } } = mongoose = require('mongoose');

const uri = 'mongodb://localhost/looktest';

mongoose.Promise = global.Promise;
mongoose.set('debug', true);

const itemTagSchema = new Schema({
  tagName: String
});

const itemSchema = new Schema({
  title: String,
  description: String,
  tags: [{ type: Schema.Types.ObjectId, ref: 'ItemTag' }]
},{ timestamps: true });

itemSchema.statics.lookup = function({ path, query }) {
  let rel =
    mongoose.model(this.schema.path(path).caster.options.ref);

  // MongoDB 3.6 and up $lookup with sub-pipeline
  let pipeline = [
    { "$lookup": {
      "from": rel.collection.name,
      "as": path,
      "let": { [path]: `$${path}` },
      "pipeline": [
        { "$match": {
          ...query,
          "$expr": { "$in": [ "$_id", `$$${path}` ] }
        }}
      ]
    }}
  ];

  return this.aggregate(pipeline).exec().then(r => r.map(m =>
    this({ ...m, [path]: m[path].map(r => rel(r)) })
  ));
};

const Item = mongoose.model('Item', itemSchema);
const ItemTag = mongoose.model('ItemTag', itemTagSchema);

const log = body => console.log(JSON.stringify(body, undefined, 2));

(async function() {

  try {

    const conn = await mongoose.connect(uri);

    // Clean data
    await Promise.all(Object.entries(conn.models).map(([k,m]) => m.remove()));

    // Create tags and items
    const tags = await ItemTag.insertMany(
      ["movies", "funny"].map(tagName => ({ tagName }))
    );

    const item = await Item.create({
      "title": "Something",
      "description": "An item",
      tags
    });

    // Query with our static
    let result = (await Item.lookup({
      path: 'tags',
      query: { 'tagName': { '$in': [ 'funny', 'politics' ] } }
    })).pop();
    log(result);


    await mongoose.disconnect();

  } catch(e) {
    console.error(e)
  } finally {
    process.exit()
  }

})()

回答by aaronheckmann

what you are asking for isn't directly supported but can be achieved by adding another filter step after the query returns.

您所要求的不是直接支持的，但可以通过在查询返回后添加另一个过滤步骤来实现。

first, .populate( 'tags', null, { tagName: { $in: ['funny', 'politics'] } } )is definitely what you need to do to filter the tags documents. then, after the query returns you'll need to manually filter out documents that don't have any tagsdocs that matched the populate criteria. something like:

首先，.populate( 'tags', null, { tagName: { $in: ['funny', 'politics'] } } )绝对是您过滤标签文档所需要做的。然后，在查询返回后，您需要手动过滤掉没有任何tags符合填充条件的文档的文档。就像是：

query....
.exec(function(err, docs){
   docs = docs.filter(function(doc){
     return doc.tags.length;
   })
   // do stuff with docs
});

回答by Aafreen Sheikh

Try replacing

尝试更换

.populate('tags').where('tags.tagName').in(['funny', 'politics']) 

by

经过

.populate( 'tags', null, { tagName: { $in: ['funny', 'politics'] } } )

回答by Fabian

Update: Please take a look at the comments - this answer does not correctly match to the question, but maybe it answers other questions of users which came across (I think that because of the upvotes) so I will not delete this "answer":

更新：请查看评论 - 这个答案与问题不正确匹配，但也许它回答了用户遇到的其他问题（我认为这是因为投票）所以我不会删除这个“答案”：

First: I know this question is really outdated, but I searched for exactly this problem and this SO post was the Google entry #1. So I implemented the docs.filterversion (accepted answer) but as I read in the mongoose v4.6.0 docswe can now simply use:

第一：我知道这个问题真的已经过时了，但我搜索了这个问题，这个 SO 帖子是谷歌条目 #1。所以我实现了这个docs.filter版本（接受的答案），但正如我在mongoose v4.6.0 文档中读到的，我们现在可以简单地使用：

Item.find({}).populate({
    path: 'tags',
    match: { tagName: { $in: ['funny', 'politics'] }}
}).exec((err, items) => {
  console.log(items.tags) 
  // contains only tags where tagName is 'funny' or 'politics'
})

Hope this helps future search machine users.

希望这有助于未来的搜索机用户。

回答by OllyBarca

After having the same problem myself recently, I've come up with the following solution:

最近我自己也遇到了同样的问题，我想出了以下解决方案：

First, find all ItemTags where tagName is either 'funny' or 'politics' and return an array of ItemTag _ids.

首先，找到 tagName 为 'funny' 或 'politics' 的所有 ItemTag，并返回一个 ItemTag _id 数组。

Then, find Items which contain all ItemTag _ids in the tags array

然后，在 tags 数组中找到包含所有 ItemTag _ids 的 Items

ItemTag
  .find({ tagName : { $in : ['funny','politics'] } })
  .lean()
  .distinct('_id')
  .exec((err, itemTagIds) => {
     if (err) { console.error(err); }
     Item.find({ tag: { $all: itemTagIds} }, (err, items) => {
        console.log(items); // Items filtered by tagName
     });
  });

回答by HernanFila

@aaronheckmann 's answerworked for me but I had to replace return doc.tags.length;to return doc.tags != null;because that field contain nullif it doesn't match with the conditions written inside populate. So the final code:

@aaronheckmann 的答案对我有用，但我不得不替换return doc.tags.length;为，return doc.tags != null;因为如果该字段与填充中写入的条件不匹配，则该字段包含null。所以最后的代码：

query....
.exec(function(err, docs){
   docs = docs.filter(function(doc){
     return doc.tags != null;
   })
   // do stuff with docs
});