Javascript Firestore 获取数据时性能缓慢的问题

Firestore slow performance issue on getting data

提问by Olivier P

I'm having slow performance issues with Firestore while retrieving basic data stored in a document compared to the realtime database with 1/10 ratio.

与具有 1/10 比率的实时数据库相比，我在检索存储在文档中的基本数据时遇到了 Firestore 性能缓慢的问题。

Using Firestore, it takes an average of 3000 ms on the first call

使用 Firestore，第一次调用平均需要 3000 毫秒

 this.db.collection(‘testCol')
   .doc(‘testDoc')
   .valueChanges().forEach((data) => {
     console.log(data);//3000 ms later
 });

Using the realtime database, it takes an average of 300 ms on the first call

使用实时数据库，首次调用平均耗时 300 毫秒

 this.db.database.ref(‘/test').once(‘value').then(data => {
     console.log(data); //300ms later
 });

This is a screenshot of the network console :

这是网络控制台的屏幕截图：

I'm running the Javascript SDK v4.50 with AngularFire2 v5.0 rc.2.

我正在使用 AngularFire2 v5.0 rc.2 运行 Javascript SDK v4.50。

Did anyone experience this issue ?

有没有人遇到过这个问题？

回答by Saul

UPDATE: 12th Feb 2018 - iOS Firestore SDK v0.10.0

更新：2018 年 2 月 12 日 - iOS Firestore SDK v0.10.0

Similar to some other commenters, I've also noticed a slower response on the first get request (with subsequent requests taking ~100ms). For me it's not as bad as 30s, but maybe around 2-3s when I have good connectivity, which is enough to provide a bad user experience when my app starts up.

与其他一些评论者类似，我也注意到第一个 get 请求的响应较慢（后续请求需要大约 100 毫秒）。对我来说，它没有 30 秒那么糟糕，但是当我有良好的连接时可能会在 2-3 秒左右，这足以在我的应用程序启动时提供糟糕的用户体验。

Firebase have advised that they're aware of this "cold start" issue and they're working on a long term fix for it - no ETA unfortunately. I think it's a separate issue that when I have poor connectivity, it can take ages (over 30s) before get requests decide to read from cache.

Firebase 表示他们已经意识到这个“冷启动”问题，并且正在为此进行长期修复 - 不幸的是没有 ETA。我认为这是一个单独的问题，当我的连接性较差时，在 get 请求决定从缓存中读取之前可能需要很长时间（超过 30 秒）。

Whilst Firebase fix all these issues, I've started using the new disableNetwork()and enableNetwork()methods (available in Firestore v0.10.0) to manually control the online/offline state of Firebase. Though I've had to be verycareful where I use it in my code, as there's a Firestore bug that can cause a crash under certain scenarios.

虽然火力地堡修复所有这些问题，我一直在使用新开工disableNetwork()和enableNetwork()方法（在公司的FireStore v0.10.0可用），手动控制火力地堡的在线/离线状态。尽管我在代码中使用它时必须非常小心，因为在某些情况下有一个 Firestore 错误可能会导致崩溃。

---

---

UPDATE: 15th Nov 2017 - iOS Firestore SDK v0.9.2

更新：2017 年 11 月 15 日 - iOS Firestore SDK v0.9.2

It seems the slow performance issue has now been fixed. I've re-run the tests described below and the time it takes for Firestore to return the 100 documents now seems to be consistently around 100ms.

看来性能缓慢的问题现在已经解决了。我重新运行了下面描述的测试，现在 Firestore 返回 100 个文档所需的时间似乎一直在 100 毫秒左右。

Not sure if this was a fix in the latest SDK v0.9.2 or if it was a backend fix (or both), but I suggest everyone updates their Firebase pods. My app is noticeably more responsive - similar to the way it was on the Realtime DB.

不确定这是最新 SDK v0.9.2 中的修复还是后端修复（或两者），但我建议每个人都更新他们的 Firebase pod。我的应用程序的响应速度明显更快 - 类似于它在实时数据库上的方式。

---

---

I've also discovered Firestore to be much slower than Realtime DB, especially when reading from lots of documents.

我还发现 Firestore 比实时数据库慢得多，尤其是在读取大量文档时。

Updated tests (with latest iOS Firestore SDK v0.9.0):

更新的测试（使用最新的 iOS Firestore SDK v0.9.0）：

I set up a test project in iOS Swift using both RTDB and Firestore and ran 100 sequential read operations on each. For the RTDB, I tested the observeSingleEvent and observe methods on each of the 100 top level nodes. For Firestore, I used the getDocument and addSnapshotListener methods at each of the 100 documents in the TestCol collection. I ran the tests with disk persistence on and off. Please refer to the attached image, which shows the data structure for each database.

我使用 RTDB 和 Firestore 在 iOS Swift 中设置了一个测试项目，并在每个项目上运行了 100 次顺序读取操作。对于 RTDB，我在 100 个顶级节点中的每一个上测试了 observeSingleEvent 和 observe 方法。对于 Firestore，我在 TestCol 集合中的 100 个文档中的每个文档中使用了 getDocument 和 addSnapshotListener 方法。我在打开和关闭磁盘持久性的情况下运行测试。请参阅附图，其中显示了每个数据库的数据结构。

I ran the test 10 times for each database on the same device and a stable wifi network. Existing observers and listeners were destroyed before each new run.

我在同一台设备和稳定的 wifi 网络上对每个数据库运行了 10 次测试。在每次新运行之前，现有的观察者和侦听器都会被销毁。

Realtime DB observeSingleEvent method:

实时数据库observeSingleEvent 方法：

func rtdbObserveSingle() {

    let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))
    print("Started reading from RTDB at: \(start)")

    for i in 1...100 {
        Database.database().reference().child(String(i)).observeSingleEvent(of: .value) { snapshot in
            let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))
            let data = snapshot.value as? [String: String] ?? [:]
            print("Data: \(data). Returned at: \(time)")
        }
    }
}

Realtime DB observe method:

实时数据库观察方法：

func rtdbObserve() {

    let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))
    print("Started reading from RTDB at: \(start)")

    for i in 1...100 {
        Database.database().reference().child(String(i)).observe(.value) { snapshot in
            let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))
            let data = snapshot.value as? [String: String] ?? [:]
            print("Data: \(data). Returned at: \(time)")
        }
    }
}

Firestore getDocument method:

Firestore getDocument 方法：

func fsGetDocument() {

    let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))
    print("Started reading from FS at: \(start)")

    for i in 1...100 {
        Firestore.firestore().collection("TestCol").document(String(i)).getDocument() { document, error in

            let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))
            guard let document = document, document.exists && error == nil else {
                print("Error: \(error?.localizedDescription ?? "nil"). Returned at: \(time)")
                return
            }
            let data = document.data() as? [String: String] ?? [:]
            print("Data: \(data). Returned at: \(time)")
        }
    }
}

Firestore addSnapshotListener method:

Firestore addSnapshotListener 方法：

func fsAddSnapshotListener() {

    let start = UInt64(floor(Date().timeIntervalSince1970 * 1000))
    print("Started reading from FS at: \(start)")

    for i in 1...100 {
        Firestore.firestore().collection("TestCol").document(String(i)).addSnapshotListener() { document, error in

            let time = UInt64(floor(Date().timeIntervalSince1970 * 1000))
            guard let document = document, document.exists && error == nil else {
                print("Error: \(error?.localizedDescription ?? "nil"). Returned at: \(time)")
                return
            }
            let data = document.data() as? [String: String] ?? [:]
            print("Data: \(data). Returned at: \(time)")
        }
    }
}

Each method essentially prints the unix timestamp in milliseconds when the method starts executing and then prints another unix timestamp when each read operation returns. I took the difference between the initial timestamp and the last timestamp to return.

每个方法本质上在方法开始执行时以毫秒为单位打印 unix 时间戳，然后在每个读取操作返回时打印另一个 unix 时间戳。我取了初始时间戳和最后一个时间戳之间的差异来返回。

RESULTS - Disk persistence disabled:

结果 - 磁盘持久性禁用：

RESULTS - Disk persistence enabled:

结果 - 启用磁盘持久性：

Data Structure:

数据结构：

When the Firestore getDocument / addSnapshotListener methods get stuck, it seems to get stuck for durations that are roughly multiples of 30 seconds. Perhaps this could help the Firebase team isolate where in the SDK it's getting stuck?

当 Firestore getDocument / addSnapshotListener 方法卡住时，它似乎卡住了大约 30 秒的倍数的持续时间。也许这可以帮助 Firebase 团队隔离它在 SDK 中卡住的位置？

回答by Terrence

Update Date March 02, 2018

更新日期 2018 年 3 月 2 日

It looks like this is a known issue and the engineers at Firestore are working on a fix. After a few email exchanges and code sharing with a Firestore engineer on this issue, this was his response as of today.

看起来这是一个已知问题，Firestore 的工程师正在努力修复。在与 Firestore 工程师就此问题进行了几次电子邮件交流和代码共享后，这是他今天的回应。

"You are actually correct. Upon further checking, this slowness on getDocuments() API is a known behavior in Cloud Firestore beta. Our engineers are aware of this performance issue tagged as "cold starts", but don't worry as we are doing our best to improve Firestore query performance.

We are already working on a long-term fix but I can't share any timelines or specifics at the moment. While Firestore is still on beta, expect that there will be more improvements to come."

“您实际上是正确的。经过进一步检查，getDocuments() API 的这种缓慢是 Cloud Firestore 测试版中的一个已知行为。我们的工程师知道这个标记为“冷启动”的性能问题，但不要担心，因为我们正在做我们尽最大努力提高 Firestore 查询性能。

我们已经在进行长期修复，但目前我无法分享任何时间表或细节。虽然 Firestore 仍处于测试阶段，但预计会有更多改进。”

So hopefully this will get knocked out soon.

所以希望这会很快被淘汰。

---

---

Using Swift / iOS

使用 Swift / iOS

After dealing with this for about 3 days it seems the issue is definitely the get() ie .getDocuments and .getDocument. Things I thoughtwere causing the extreme yet intermittent delays but don't appear to be the case:

在处理了大约 3 天之后，似乎问题肯定是 get() 即 .getDocuments 和 .getDocument。我认为导致极端但间歇性延迟的事情，但似乎并非如此：

1. Not so great network connectivity
2. Repeated calls via looping over .getDocument()
3. Chaining get() calls
4. Firestore Cold starting
5. Fetching multiple documents (Fetching 1 small doc caused 20sec delays)
6. Caching (I disabled offline persistence but this did nothing.)

1. 不是很好的网络连接
2. 通过循环 .getDocument() 重复调用
3. 链接 get() 调用
4. Firestore 冷启动
5. 获取多个文档（获取 1 个小文档导致 20 秒延迟）
6. 缓存（我禁用了离线持久性，但这什么也没做。）

I was able to rule all of these out as I noticed this issue didn't happen with every Firestore database call I was making. Only retrievals using get(). For kicks I replaced .getDocument with .addSnapshotListener to retrieve my data and voila. Instant retrieval each time including the first call. No cold starts. So far no issues with the .addSnapshotListener, only getDocument(s).

我能够排除所有这些，因为我注意到这个问题并没有发生在我进行的每个 Firestore 数据库调用中。仅检索使用 get()。为了踢球，我用 .addSnapshotListener 替换了 .getDocument 来检索我的数据，瞧。每次即时检索，包括第一次通话。没有冷启动。到目前为止，.addSnapshotListener 没有问题，只有 getDocument(s)。

For now, I'm simply dropping the .getDocument() where time is of the essence and replacing it with .addSnapshotListener then using

现在，我只是删除时间至关重要的 .getDocument() 并将其替换为 .addSnapshotListener 然后使用

for document in querySnapshot!.documents{
// do some magical unicorn stuff here with my document.data()
}

... in order to keep moving until this gets worked out by Firestore.

...为了继续前进，直到 Firestore 解决了这个问题。

回答by Hendies

I had this issue until this morning. My Firestore query via iOS/Swift would take around 20 seconds to complete a simple, fully indexed query - with non-proportional query times for 1 item returned - all the way up to 3,000.

直到今天早上我才遇到这个问题。我通过 iOS/Swift 的 Firestore 查询需要大约 20 秒才能完成一个简单的、完全索引的查询 - 返回 1 个项目的非比例查询时间 - 一直到 3,000。

My solution was to disable offline data persistence. In my case, it didn't suit the needs of our Firestore database - which has large portions of its data updated every day.

我的解决方案是禁用离线数据持久性。就我而言，它不适合我们 Firestore 数据库的需求——它的大部分数据每天都在更新。

iOS & Android users have this option enabled by default, whilst web users have it disabled by default. It makes Firestore seem insanely slow if you're querying a huge collection of documents. Basically it caches a copy of whichever data you're querying (and whichever collection you're querying - I believe it caches all documents within) which can lead to high Memory usage.

iOS 和 Android 用户默认启用此选项，而网络用户默认禁用此选项。如果您正在查询大量文档，它会使 Firestore 看起来非常慢。基本上它会缓存您正在查询的任何数据的副本（以及您正在查询的任何集合 - 我相信它会缓存其中的所有文档），这可能会导致内存使用率过高。

In my case, it caused a huge wait for every query until the device had cached the data required - hence the non-proportional query times for the increasing numbers of items to return from the exact same collection. This is because it took the same amount of time to cache the collection in each query.

就我而言，它导致每次查询都需要大量等待，直到设备缓存了所需的数据 - 因此，从完全相同的集合中返回的项目数量不断增加，查询时间不成比例。这是因为在每个查询中缓存集合花费了相同的时间。

Offline Data - from the Cloud Firestore Docs

离线数据 - 来自 Cloud Firestore Docs

I performed some benchmarking to display this effect (with offline persistence enabled) from the same queried collection, but with different amounts of items returned using the .limit parameter:

我执行了一些基准测试来显示来自同一个查询集合的这种效果（启用离线持久性），但使用 .limit 参数返回不同数量的项目：

Now at 100 items returned (with offline persistence disabled), my query takes less than 1 second to complete.

现在返回 100 个项目（禁用离线持久性），我的查询不到 1 秒即可完成。

My Firestore query code is below:

我的 Firestore 查询代码如下：

let db = Firestore.firestore()
self.date = Date()
let ref = db.collection("collection").whereField("Int", isEqualTo: SomeInt).order(by: "AnotherInt", descending: true).limit(to: 100)
ref.getDocuments() { (querySnapshot, err) in
    if let err = err {
        print("Error getting documents: \(err)")
    } else {
        for document in querySnapshot!.documents {
            let data = document.data()
            //Do things
        }
        print("QUERY DONE")
        let currentTime = Date()
        let components = Calendar.current.dateComponents([.second], from: self.date, to: currentTime)
        let seconds = components.second!
        print("Elapsed time for Firestore query -> \(seconds)s")
        // Benchmark result
    }
}

回答by Kyo Kurosagi

well, from what I'm currently doing and research by using nexus 5X in emulator and real android phone Huawei P8,

好吧，从我目前在模拟器和真正的安卓手机华为 P8 中使用 nexus 5X 所做的和研究，

Firestore and Cloud Storage are both give me a headache of slow response when I do first document.get() and first storage.getDownloadUrl()

当我第一次执行 document.get() 和第一次 storage.getDownloadUrl() 时，Firestore 和 Cloud Storage 都让我头疼

It give me more than 60 seconds response on each request. The slow response only happen in real android phone. Not in emulator. Another strange thing. After the first encounter, the rest request is smooth.

它给我每个请求超过 60 秒的响应。缓慢的响应只发生在真正的安卓手机中。不在模拟器中。另一个奇怪的事情。第一次相遇后，其余请求顺利。

Here is the simple code where I meet the slow response.

这是我遇到响应缓慢的简单代码。

var dbuserref = dbFireStore.collection('user').where('email','==',email);
const querySnapshot = await dbuserref.get();

var url = await defaultStorage.ref(document.data().image_path).getDownloadURL();

I also found link that is researching the same. https://reformatcode.com/code/android/firestore-document-get-performance

我还发现正在研究相同的链接。 https://reformatcode.com/code/android/firestore-document-get-performance

回答by JPJ

Almost 3 years later, firestore being well out of beta and I can confirm that this horrible problem still persists ;-(

差不多 3 年后，firestore 已经完全退出测试版，我可以确认这个可怕的问题仍然存在；-(

On our mobile app we use the javascript / node.js firebase client. After a lot of testing to find out why our app's startup time is around 10sec we identified what to attribute 70% of that time to... Well, to firebase's and firestore's performance and cold start issues:

在我们的移动应用程序中，我们使用 javascript / node.js firebase 客户端。在经过大量测试以找出为什么我们的应用程序的启动时间约为 10 秒后，我们确定了将 70% 的时间归因于……嗯，firebase 和 firestore 的性能和冷启动问题：

• firebase.auth().onAuthStateChanged() fires approx. after 1.5 - 2sec, already quite bad.
• If it returns a user, we use its ID to get the user document from firestore. This is the first call to firestore and the corresponding get() takes 4 - 5sec. Subsequent get() of the same or other documents take approx. 500ms.

• firebase.auth().onAuthStateChanged() 大约触发。1.5 - 2 秒后，已经很糟糕了。
• 如果它返回一个用户，我们使用它的 ID 从 firestore 获取用户文档。这是对 firestore 的第一次调用，相应的 get() 需要 4 - 5 秒。相同或其他文档的后续 get() 需要大约。500 毫秒。

So in total the user initialization takes 6 - 7 sec, completely unacceptable. And we can't do anything about it. We can't test disabling persistence, since in the javascript client there's no such option, persistence is always enabled by default, so not calling enablePersistence() won't change anything.

因此，用户初始化总共需要 6 - 7 秒，这是完全不可接受的。我们对此无能为力。我们无法测试禁用持久性，因为在 javascript 客户端中没有这样的选项，默认情况下始终启用持久性，因此不调用 enablePersistence() 不会改变任何内容。